go.mod: Update dependencies

This implements HTTP basicauth into Caddy 2. The basic auth module will
not work with passwords that are not securely hashed, so a subcommand
hash-password was added to make it convenient to produce those hashes.

Also included is Caddyfile support.

Closes #2747.

.gitignore

@@ -1,12 +1,17 @@
-.DS_Store
-Thumbs.db
 _gitignore/
-Vagrantfile
-.vagrant/
-dist/
-
-error.log
-access.log
-
-/*.conf
+*.log
 Caddyfile
+
+# artifacts from pprof tooling
+*.prof
+*.test
+
+# build artifacts
+cmd/caddy/caddy
+cmd/caddy/caddy.exe
+
+# mac specific
+.DS_Store
+
+# go modules
+vendor

.travis.yml

@@ -1,2 +0,0 @@
-language: go
-script: go test ./...

AUTHORS

@@ -0,0 +1,10 @@
+# This is the official list of Caddy Authors for copyright purposes.
+# Authors may be either individual people or legal entities.
+#
+# Not all individual contributors are authors. For the full list of
+# contributors, refer to the project's page on GitHub or the repo's
+# commit history.
+
+Matthew Holt <Matthew.Holt@gmail.com>
+Light Code Labs <sales@lightcodelabs.com>
+Ardan Labs <info@ardanlabs.com>

CONTRIBUTING.md

@@ -1,32 +0,0 @@
-## Contributing to Caddy
-
-**[Join us on Slack](https://gophers.slack.com/messages/caddy/)** to chat with other Caddy developers! ([Request an invite](http://bit.ly/go-slack-signup), then join the #caddy channel.)
-
-This project gladly accepts contributions and we encourage interested users to get involved!
-
-
-#### For small tweaks, bug fixes, and tests
-
-Submit [pull requests](https://github.com/mholt/caddy/pulls) at any time. Thank you for helping out in simple ways! Bug fixes should be under test to assert correct behavior.
-
-
-#### Ideas, questions, bug reports
-
-You should totally [open an issue](https://github.com/mholt/caddy/issues) with your ideas, questions, and bug reports, if one does not already exist for it. Bug reports should state expected behavior and contain clear instructions for reproducing the problem.
-
-
-#### New features
-
-Before submitting a pull request, please open an issue first to discuss it and claim it. This prevents overlapping efforts and keeps the project in-line with its goals. If you prefer to discuss the feature privately, you can reach other developers on Slack or you may email me directly. (My email address is below.)
-
-And don't forget to write tests for new features!
-
-
-#### Vulnerabilities
-
-If you've found a vulnerability that is serious, please email me: Matthew dot Holt at Gmail. If it's not a big deal, a pull request will probably be faster.
-
-
-## Thank you
-
-Thanks for your help! Caddy would not be what it is today without your contributions.

LICENSE

@@ -1,3 +1,4 @@
+
                                  Apache License
                            Version 2.0, January 2004
                         http://www.apache.org/licenses/
@@ -178,7 +179,7 @@
    APPENDIX: How to apply the Apache License to your work.
 
       To apply the Apache License to your work, attach the following
-      boilerplate notice, with the fields enclosed by brackets "{}"
+      boilerplate notice, with the fields enclosed by brackets "[]"
       replaced with your own identifying information. (Don't include
       the brackets!)  The text should be enclosed in the appropriate
       comment syntax for the file format. We also recommend that a
@@ -186,7 +187,7 @@
       same "printed page" as the copyright notice for easier
       identification within third-party archives.
 
-   Copyright {yyyy} {name of copyright owner}
+   Copyright [yyyy] [name of copyright owner]
 
    Licensed under the Apache License, Version 2.0 (the "License");
    you may not use this file except in compliance with the License.

README.md

@@ -1,125 +1,380 @@
-[![Caddy](https://caddyserver.com/resources/images/caddy-boxed.png)](https://caddyserver.com)
+Caddy 2 Development Branch
+===========================
 
-[![Documentation](https://img.shields.io/badge/godoc-reference-blue.svg?style=flat-square)](https://godoc.org/github.com/mholt/caddy) [![Build Status](https://img.shields.io/travis/mholt/caddy.svg?style=flat-square)](https://travis-ci.org/mholt/caddy)
+[![Build Status](https://dev.azure.com/mholt-dev/Caddy/_apis/build/status/Multiplatform%20Tests?branchName=v2)](https://dev.azure.com/mholt-dev/Caddy/_build/latest?definitionId=5&branchName=v2)
 
-Caddy is a lightweight, general-purpose web server for Windows, Mac, Linux, BSD, and [Android](https://github.com/mholt/caddy/wiki/Running-Caddy-on-Android). It is a capable alternative to other popular web servers that is easy to use.
-
-The most notable features are HTTP/2, Virtual Hosts, TLS + SNI, and easy configuration with a [Caddyfile](https://caddyserver.com/docs/caddyfile). Usually, you have one Caddyfile per site. Most directives for the Caddyfile invoke a layer of middleware which can be [used in your own Go programs](https://github.com/mholt/caddy/wiki/Using-Caddy-Middleware-in-Your-Own-Programs).
-
-[Download](https://github.com/mholt/caddy/releases) · [User Guide](https://caddyserver.com/docs)
+This is the development branch for Caddy 2. This code (version 2) is not yet feature-complete or production-ready, but is already being used in production, and we encourage you to deploy it today on sites that are not very visible or important so that it can obtain crucial experience in the field.
 
+Please file issues to propose new features and report bugs, and after the bug or feature has been discussed, submit a pull request! We need your help to build this web server into what you want it to be. (Caddy 2 issues and pull requests receive priority over Caddy 1 issues and pull requests.)
 
+**Caddy 2 is the web server of the Go community.** We are looking for maintainers to represent the community! Please become involved (issues, PRs, [our forum](https://caddy.community) etc.) and express interest if you are committed to being a collaborator on the Caddy project.
 
 
 ### Menu
 
-- [Getting Caddy](#getting-caddy)
-- [Running from Source](#running-from-source)
+- [Build from source](#build-from-source)
 - [Quick Start](#quick-start)
-- [Contributing](#contributing)
-- [About the Project](#about-the-project)
+- [Configuration](#configuration)
+- [Full Documentation](#full-documentation)
+- [List of Improvements](#list-of-improvements)
+- [FAQ](#faq)
 
 
+## Build from source
 
+Requirements:
 
-## Getting Caddy
+- [Go 1.13 or newer](https://golang.org/dl/)
+- Make sure you do not disable [Go modules](https://github.com/golang/go/wiki/Modules) (`export GO111MODULE=auto`)
 
-Caddy binaries have no dependencies and are available for nearly every platform.
+Download the `v2` source code:
 
-[Latest release](https://github.com/mholt/caddy/releases/latest)
+```bash
+$ git clone -b v2 "https://github.com/caddyserver/caddy.git"
+```
 
+Build:
 
-## Running from Source
-
-Note: You will need **[Go 1.4](https://golang.org/dl)** or newer
-
-1. `$ go get github.com/mholt/caddy`
-2. `cd` into your website's directory
-3. Run `caddy` (assumes `$GOPATH/bin` is in your `$PATH`)
-
-If you're tinkering, you can also use `go run main.go`.
-
-By default, Caddy serves the current directory at [localhost:2015](http://localhost:2015). You can place a Caddyfile to configure Caddy for serving your site.
-
-Caddy accepts some flags from the command line. Run `caddy -h` to view the help for flags. You can also pipe a Caddyfile into the caddy command.
-
-
-
-#### Docker Container
-
-Caddy is [available as a Docker container](https://registry.hub.docker.com/u/darron/caddy/).
-
-
-
-#### 3rd-party libraries
-
-Although Caddy's binaries are completely static, Caddy relies on some excellent libraries. [Godoc.org](https://godoc.org/github.com/mholt/caddy) shows the packages that each Caddy package imports.
+```bash
+$ cd caddy/cmd/caddy/
+$ go build
+```
 
+That will put a `caddy(.exe)` binary into the current directory. You can move it into your PATH or use `go install` to do that automatically (assuming `$GOPATH/bin` is already in your PATH). You can also use `go run main.go` for quick, temporary builds while developing.
 
+The initial build may be slow as dependencies are downloaded. Subsequent builds should be very fast. If you encounter any Go-module-related errors, try clearing your Go module cache (`$GOPATH/pkg/mod`) and Go package cache (`$GOPATH/pkg`) and read [the Go wiki page about modules for help](https://github.com/golang/go/wiki/Modules). If you have issues with Go modules, please consult the Go community for help. But if there is an actual error in Caddy, please report it to us.
 
 
 ## Quick Start
 
-The website has [full documentation](https://caddyserver.com/docs) but this will get you started in about 30 seconds:
+(Until the stable 2.0 release, there may be breaking changes in v2, please be aware!)
 
-Place a file named "Caddyfile" with your site. Paste this into it and save:
+These instructions assume an executable build of Caddy 2 is named `caddy` in the current folder. If it's in your PATH, you may omit the path to the binary (`./`).
 
-```
-localhost
+Start Caddy:
 
-gzip
-browse
-ext .html
-websocket /echo cat
-log ../access.log
-header /api Access-Control-Allow-Origin *
+```bash
+$ ./caddy start
 ```
 
-Run `caddy` from that directory, and it will automatically use that Caddyfile to configure itself.
-
-That simple file enables compression, allows directory browsing (for folders without an index file), serves clean URLs, hosts an echo server for WebSocket connections at /echo, logs accesses to access.log, and adds the coveted `Access-Control-Allow-Origin: *` header for all responses from some API.
-
-Wow! Caddy can do a lot with just a few lines.
-
-#### Defining multiple sites
-
-You can run multiple sites from the same Caddyfile, too:
+There are no config files with Caddy 2. Instead, you POST configuration to it:
 
+```bash
+$ curl -X POST "http://localhost:2019/load" \
+    -H "Content-Type: application/json" \
+    -d @- << EOF
+    {
+        "apps": {
+            "http": {
+                "servers": {
+                    "example": {
+                        "listen": ["127.0.0.1:2080"],
+                        "routes": [
+                            {
+                                "handle": [{
+                                    "handler": "file_server",
+                                    "browse": {}
+                                }]
+                            }
+                        ]
+                    }
+                }
+            }
+        }
+    }
+EOF
 ```
-http://mysite.com,
-http://www.mysite.com {
-	redir https://mysite.com
-}
 
-https://mysite.com {
-	tls mysite.crt mysite.key
-	# ...
+Now visit http://localhost:2080 in your browser and you will see the contents of the current directory displayed.
+
+To change Caddy's configuration, simply POST a new payload to that endpoint. Config changes are extremely lightweight and efficient, and should be graceful on all platforms -- _even Windows_.
+
+Updating configuration using heredoc can be tedious, so you can still use a config file if you prefer. Put your configuration in any file (`caddy.json` for example) and then POST that instead:
+
+```bash
+$ curl -X POST "http://localhost:2019/load" \
+    -H "Content-Type: application/json" \
+    -d @caddy.json
+```
+
+Or you can tell Caddy to load its configuration from a file in the first place (this simply does the work of the above curl command for you):
+
+```bash
+$ ./caddy start --config caddy.json
+```
+
+To stop Caddy:
+
+```bash
+$ ./caddy stop
+```
+
+Note that this will stop any process named the same as `os.Args[0]`.
+
+For other commands, please see [the Caddy 2 documentation](https://github.com/caddyserver/caddy/wiki/v2:-Documentation).
+
+### Caddyfile
+
+Caddy 2 can be configured with a Caddyfile, much like in v1, for example:
+
+```plain
+example.com
+
+templates
+encode    gzip zstd
+try_files {path}.html {path}
+reverse_proxy /api localhost:9005
+file_server
+```
+
+Instead of being its primary mode of configuration, an internal _config adapter_ adapts the Caddyfile to Caddy's native JSON structure. You can see it in action with the [`adapt` command](https://github.com/caddyserver/caddy/wiki/v2:-Documentation#adapt):
+
+```bash
+$ ./caddy adapt --config path/to/Caddyfile --adapter caddyfile --pretty
+```
+
+If you just want to run Caddy with your Caddyfile directly, the CLI wraps this up for you nicely. Either of the following commands:
+
+```bash
+$ ./caddy start
+$ ./caddy run
+```
+
+will use your Caddyfile if it is called `Caddyfile` in the current directory.
+
+If your Caddyfile is somewhere else, you can still use it:
+
+```bash
+$ ./caddy start|run --config path/to/Caddyfile --adapter caddyfile
+```
+
+[Learn more about the Caddyfile in v2.](https://github.com/caddyserver/caddy/wiki/v2:-Documentation#caddyfile-adapter)
+
+
+## Configuration
+
+Caddy 2 exposes an unprecedented level of control compared to any web server in existence. In Caddy 2, you are usually setting the actual values of the initialized types in memory that power everything from your HTTP handlers and TLS handshakes to your storage medium. Caddy 2 is also ridiculously extensible, with a module system that makes vast improvements over Caddy 1's plugin system.
+
+Nearly all of Caddy 2's configuration is contained in a single config document, rather than being spread across CLI flags and env variables and a configuration file as with other web servers (and Caddy 1).
+
+To wield the power of this design, you need to know how the config document is structured. Please see the [the Caddy 2 documentation in our wiki](https://github.com/caddyserver/caddy/wiki/v2:-Documentation) for details about Caddy's config structure.
+
+Configuration is normally given to Caddy through an API endpoint, which is likewise documented in the wiki pages. However, you can also use config files of various formats with [config adapters](https://github.com/caddyserver/caddy/wiki/v2:-Documentation#config-adapters).
+
+
+## Full Documentation
+
+Caddy 2 is very much in development, so the documentation is an ongoing WIP, but the latest will be in our wiki for now:
+
+**https://github.com/caddyserver/caddy/wiki/v2:-Documentation**
+
+Note that breaking changes are expected until the stable 2.0 release.
+
+
+## List of Improvements
+
+The following is a non-comprehensive list of significant improvements over Caddy 1. Not everything in this list is finished yet, but they will be finished or at least will be possible with Caddy 2:
+
+- Centralized configuration. No more disparate use of environment variables, config files (potentially multiple!), CLI flags, etc.
+- REST API. Control Caddy with HTTP requests to an administration endpoint. Changes are applied immediately and efficiently.
+- Dynamic configuration. Any and all specific config values can be modified directly through the admin API with a REST endpoint.
+    - Change only specific configuration settings instead of needing to specify the whole config each time. This makes it safe and easy to change Caddy's config with manually-crafted curl commands, for example.
+- No configuration files. Except optionally to bootstrap its configuration at startup. You can still use config files if you wish, and we expect that most people will.
+- Export the current Caddy configuration with an API GET request.
+- Silky-smooth graceful reloads. Update the configuration up to dozens of times per second with no dropped requests and very little memory cost. Our unique graceful reload technology is lighter and faster **and works on all platforms, including Windows**.
+- An embedded scripting language! Caddy2 has native Starlark integration. Do things you never thought possible with higher performance than Lua, JavaScript, and other VMs. Starlark is expressive, familiar (dialect of Python), _almost_ Turing-complete, and highly efficient. (We're still improving performance here.)
+- Using [XDG standards](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html#variables) instead of dumping all assets in `$HOME/.caddy`.
+- Caddy plugins are now called "Caddy modules" (although the terms "plugin" and "module" may be used interchangably). Caddy modules are a concept unrelated [Go modules](https://github.com/golang/go/wiki/Modules), except that Caddy modules may be implemented by Go modules. Caddy modules are centrally-registered, properly namespaced, and generically loaded & configured, as opposed to how scattered and unorganized Caddy 1-era plugins are.
+- Modules are easier to write, since they do not have to both deserialize their own configuration from a configuration DSL and provision themselves like plugins did. Modules are initialized pre-configured and have the ability to validate the configuration and perform provisioning steps if necessary.
+- Can specify different storage mechanisms in different parts of the configuration, if more than one is needed.
+- "Top-level" Caddy modules are simply called "apps" because literally any long-lived application can be served by Caddy 2.
+- Even more of Caddy is made of modules, allowing for unparalleled extensibility, flexibility, and control. Caddy 2 is arguably the most flexible, extensible, programmable web server ever made.
+- TLS improvements!
+	- TLS configuration is now centralized and decoupled from specific sites
+	- A single certificate cache is used process-wide, reducing duplication and improving memory use
+	- Customize how to manage each certificate ("automation policies") based on the hostname
+	- Automation policy doesn't have to be limited to just ACME - could be any way to manage certificates
+	- Fine-grained control over TLS handshakes
+	- If an ACME challenge fails, other enabled challenges will be tried (no other web server does this)
+    - TLS Session Ticket Ephemeral Keys (STEKs) can be rotated in a cluster for increased performance (no other web server does this either!)
+    - Ability to select a specific certificate per ClientHello given multiple qualifying certificates
+    - Provide TLS certificates without persisting them to disk; keep private keys entirely in memory
+- All-new HTTP server core
+	- Listeners can be configured for any network type, address, and port range
+	- Customizable TLS connection policies
+	- HTTP handlers are configured by "routes" which consist of matcher and handler components. Match matches an HTTP request, and handle defines the list of handlers to invoke as a result of the match.
+	- Some matchers are regular expressions, which expose capture groups to placeholders.
+	- New matchers include negation and matching based on remote IP address / CIDR ranges.
+    - Placeholders are vastly improved generally
+	- Placeholders (variables) are more properly namespaced.
+	- Multiple routes may match an HTTP request, creating a "composite route" quickly on the fly.
+	- The actual handler for any given request is its composite route.
+	- User defines the order of middlewares (careful! easy to break things).
+	- Adding middlewares no longer requires changes to Caddy's code base (there is no authoritative list).
+	- Routes may be marked as terminal, meaning no more routes will be matched.
+	- Routes may be grouped so that only the first matching route in a group is applied.
+	- Requests may be "re-handled" if they are modified and need to be sent through the chain again (internal redirect).
+	- Vastly more powerful static file server, with native content-negotiation abilities
+    - Done away with URL-rewriting hacks often needed in Caddy 1
+	- Highly descriptive/traceable errors
+	- Very flexible error handling, with the ability to specify a whole list of routes just for error cases
+	- More control over automatic HTTPS: disable entirely, disable only HTTP->HTTPS redirects, disable only cert management, and for certain names, etc.
+    - Use Starlark to build custom, dynamic HTTP handlers at request-time
+        - We are finding that -- on average -- Caddy 2's Starlark handlers are ~1.25-2x faster than NGINX+Lua.
+
+And a few major features still being worked on:
+
+- Logging
+- Kubernetes ingress controller (mostly done, just polishing it -- and it's amazing)
+- More config adapters. Caddy's native JSON config structure is powerful and complex. Config adapters upsample various formats to Caddy's native config. There are already adapters for Caddyfile, JSON 5, and JSON-C. Planned are NGINX config, YAML, and TOML. The community might be interested in building Traefik and Apache config adapters!
+
+
+
+## FAQ
+
+### How do I configure Caddy 2?
+
+Caddy's primary mode of configuration is a REST API, which accepts a JSON document. The JSON structure is described [in the wiki](https://github.com/caddyserver/caddy/wiki/v2:-Documentation). The advantages of exposing this low-level structure are 1) it has near-parity with actual memory initialization, 2) it allows us to offer wrappers over this configuration to any degree of convenience that is needed, and 3) it performs very well under rapid config changes.
+
+Basically, you will [start Caddy](https://github.com/caddyserver/caddy/wiki/v2:-Documentation#start), then [POST a JSON config to its API endpoint](https://github.com/caddyserver/caddy/wiki/v2:-Documentation#post-load).
+
+Although this makes Caddy 2 highly programmable, not everyone will want to configure Caddy via JSON with an API. Sometimes we just want to give Caddy a simple, static config file and have it do its thing. That's what **[config adapters](https://github.com/caddyserver/caddy/wiki/v2:-Documentation#config-adapters)** are for! You can configure Caddy more ways than one, depending on your needs and preferences. See the next questions that explain this more.
+
+### Caddy 2 feels harder to use. How is this an improvement over Caddy 1?
+
+Caddy's ease of use is one of the main reasons it is special. We are not taking that away in Caddy 2, but first we had to be sure to tackle the fundamental design limitations with Caddy 1. Usability can then be layered on top. This approach has several advantages which we discuss in the next question.
+
+### What about the Caddyfile; are there easier ways to configure Caddy 2?
+
+Yes! Caddy's native JSON configuration via API is nice when you are automating config changes at scale, but if you just have a simple, static configuration in a file, you can do that too with the [Caddyfile](https://github.com/caddyserver/caddy/wiki/v2:-Documentation#caddyfile-adapter).
+
+The v2 Caddyfile is very similar to the v1 Caddyfile, but they are not compatible. Several improvements have been made to request matching and directives in v2, giving you more power with less complexity and fewer inconsistencies.
+
+Caddy's default _config adapter_ is the Caddyfile adapter. This takes a Caddyfile as input and [outputs the JSON config](https://github.com/caddyserver/caddy/wiki/v2:-Documentation#adapt). You can even run Caddy directly without having to see or think about the underlying JSON config.
+
+The following _config adapters_ are already being built or plan to be built:
+
+- Caddyfile
+- JSON 5
+- JSON-C
+- nginx
+- YAML
+- TOML
+- any others that the community would like to contribute
+
+Config adapters allow you to configure Caddy not just one way but _any_ of these ways. For example, you'll be able to bring your existing NGINX config to Caddy and it will spit out the Caddy config JSON you need (to the best of its ability). How cool is that! You can then easily tweak the resulting config by hand, if necessary.
+
+All config adapters vary in their theoretical expressiveness; that is, if you need more advanced configuration you'll have to drop down to the JSON config, because the Caddyfile or an nginx config may not be expressive enough.
+
+However, we expect that most users will be able to use the Caddyfile (or another easy config adapter) exclusively for their sites.
+
+### Why JSON for configuration? Why not _&lt;any other serialization format&gt;_?
+
+We know there might be strong opinions on this one. Regardless, for Caddy 2, we've decided to go with JSON. If that proves to be a fatal mistake, then Caddy 3 probably won't use JSON.
+
+JSON may not be the fastest, the most compact, the easiest to write, serialization format that exists. But those aren't our goals. It has withstood the test of time and checks all our boxes.
+
+- It is almost entirely ubiquitous. JSON works natively in web browsers and has mature libraries in pretty much every language.
+- It is human-readable (as opposed to a binary format).
+- It is easy to tweak by hand. Although composing raw JSON by hand is not awesome, this will not be mainstream once our config adapters are done.
+- It is generally easy to convert other serializations or config formats into JSON, as opposed to the other way around.
+- Even though JSON deserialization is not fast per-se, that kind of performance is not really a concern since config reloads are not the server's hottest path like HTTP request handling or TLS handshakes are. Even with JSON, Caddy 2 can handle dozens of config changes per second, which is probably plenty for now.
+- It maps almost 1:1 to the actual, in-memory values that power your HTTP handlers and other parts of the server (no need to parse a config file with some arbitrary DSL and do a bunch of extra pre-processing).
+
+Ultimately, we think all these properties are appropriate -- if not ideal -- for a web server configuration.
+
+If you're still not happy with the choice of JSON, feel free to contribute a config adapter of your own choice!
+
+Or just use YAML or TOML, which seamlessly translate to JSON.
+
+### JSON is declarative; what if I need more programmability (i.e. imperative syntax)?
+
+NGINX also realized the need for imperative logic in declarative configs, so they tried "if" statements, [but it was a bad idea](https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/).
+
+We have good news. Caddy 2 can give you the power of imperative logic without the perils of mixing declarative and imperative config such as befell NGINX. We do this by allowing embedded imperative syntax awithin the Caddy's declarative config.
+
+Caddy 2's configuration is declarative because configuration is very much declarative in nature. Configuration is a tricky medium, as it is read and written by both computers and humans. Computers use it, but humans constantly refer to it and update it. Declarative syntaxes are fairly straightforward to make sense of, whereas it is difficult to reason about imperative logic.
+
+However, sometimes computation is useful, and in some cases, the only way to express what you need. This can be illustrated really well in the simple case of trying to decide whether a particular HTTP middleware should be invoked as part of an HTTP request. A lot of the time, such logic is as simple as: "GET requests for any path starting with /foo/bar", which can be expressed declaratively in JSON:
+
+```json
+{
+	"method": "GET",
+	"path": "/foo/bar"
 }
 ```
 
-Note that the secure host will automatically be served with HTTP/2 if the client supports it.
+But what if you need to match /foo/bar OR /topaz? How do you express that OR clause? Maybe an array:
 
-For more documentation, please view [the website](https://caddyserver.com/docs). You may also be interested in the [developer guide](https://github.com/mholt/caddy/wiki) on this project's GitHub wiki.
+```json
+{
+	"method": ["GET"],
+	"path": ["/foo/bar", "/topaz"]
+}
+```
 
+Now what if you need add a NOT or AND clause? JSON quickly tires out. As you learn about Caddy 2's request matching, you will see how we handled this. Caddy 2's JSON gives you the ability to express moderately-complex logic such as:
 
+```js
+// this is not actual Caddy config, just logic pseudocode
+IF (Host = "example.com")
+	OR (Host = "sub.example.com" AND Path != "/foo/bar")
+```
 
+Already, this is more expressive power than most web servers offer with their native config, yet Caddy 2 offers this in JSON.
 
+But in most web servers, to make logic this complex feasible, you'll generally call out to Lua or some extra DSL. For example, in NGINX you could use a Lua module to express this logic. Traefik 2.0 has [yet another kind of clunky-looking custom DSL](https://blog.containo.us/back-to-traefik-2-0-2f9aa17be305#d22e) just for this.
 
+Caddy 2 solves this in a novel way with [Starlark expressions](https://godoc.org/go.starlark.net/starlark#Eval). Starlark is a familiar dialect of Python! So, no new DSLs to learn and no VMs to slow things down:
 
-## Contributing
+```python
+req.host == 'example.com' ||
+	(req.host == 'sub.example.com' && req.path != '/foo/bar')
+```
 
-**[Join us on Slack](https://gophers.slack.com/messages/caddy/)** to chat with other Caddy developers! ([Request an invite](http://bit.ly/go-slack-signup), then join the #caddy channel.)
+Starlark performs at least as well as NGINX+Lua (more performance tests ongoing, as well as optimizations to make it even faster!) and because it's basically Python, it's familiar and easy to use.
 
-This project would not be what it is without your help. Please see the [contributing guidelines](https://github.com/mholt/caddy/blob/master/CONTRIBUTING.md) if you haven't already.
+In summary: Caddy 2 config is declarative, but can be imperative where that is useful.
 
-Thanks for making Caddy -- and the Web -- better!
+### What is Caddy 2 licensed as?
 
+Caddy 2 is licensed under the Apache 2.0 open source license. There are no official Caddy 2 distributions that are proprietary.
 
+### Does Caddy 2 have telemetry?
 
+No. There was not enough academic interest to continue supporting it. If telemetry does get added later, it will not be on by default or will be vastly reduced in its scope.
 
-## About the project
+## Does Caddy 2 use HTTPS by default?
 
-Caddy was born out of the need for a "batteries-included" web server that runs anywhere and doesn't have to take its configuration with it. Caddy took inspiration from [spark](https://github.com/rif/spark), nginx, lighttpd, Websocketd, and Vagrant, and provides a pleasant mixture of features from each of them.
+Yes. HTTPS is automatic and enabled by default when possible, just like in Caddy 1. Basically, if your HTTP routes specify a `host` matcher with qualifying domain names, those names will be enabled for automatic HTTPS. Automatic HTTPS is disabled for domains which match certificates that are manually loaded by your config.
 
+## How do I avoid Let's Encrypt rate limits with Caddy 2?
 
-*Twitter: [@mholt6](https://twitter.com/mholt6)*
+As you are testing and developing with Caddy 2, you should use test ("staging") certificates from Let's Encrypt to avoid rate limits. By default, Caddy 2 uses Let's Encrypt's production endpoint to get real certificates for your domains, but their [rate limits](https://letsencrypt.org/docs/rate-limits/) forbid testing and development use of this endpoint for good reasons. You can switch to their [staging endpoint](https://letsencrypt.org/docs/staging-environment/) by adding the staging CA to your automation policy in the `tls` app:
+
+```json
+"tls": {
+	"automation": {
+		"policies": [
+			{
+				"management": {
+					"module": "acme",
+					"ca": "https://acme-staging-v02.api.letsencrypt.org/directory"
+				}
+			}
+		]
+	}
+}
+```
+
+Or with the Caddyfile, using a global options block at the top:
+
+```
+{
+	acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
+}
+```
+
+## Can we get some access controls on the admin endpoint?
+
+Yeah, that's coming. For now, you can use a permissioned unix socket for some basic security.

admin.go

@@ -0,0 +1,280 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddy
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"io/ioutil"
+	"log"
+	"mime"
+	"net"
+	"net/http"
+	"net/http/pprof"
+	"os"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/caddyserver/caddy/v2/caddyconfig"
+	"github.com/mholt/certmagic"
+	"github.com/rs/cors"
+)
+
+var (
+	cfgEndptSrv   *http.Server
+	cfgEndptSrvMu sync.Mutex
+)
+
+// AdminConfig configures the admin endpoint.
+type AdminConfig struct {
+	Listen string `json:"listen,omitempty"`
+}
+
+// DefaultAdminConfig is the default configuration
+// for the administration endpoint.
+var DefaultAdminConfig = &AdminConfig{
+	Listen: DefaultAdminListen,
+}
+
+// StartAdmin starts Caddy's administration endpoint,
+// bootstrapping it with an optional configuration
+// in the format of JSON bytes. It opens a listener
+// resource. When no longer needed, StopAdmin should
+// be called.
+func StartAdmin(initialConfigJSON []byte) error {
+	cfgEndptSrvMu.Lock()
+	defer cfgEndptSrvMu.Unlock()
+
+	adminConfig := DefaultAdminConfig
+	if len(initialConfigJSON) > 0 {
+		var config *Config
+		err := json.Unmarshal(initialConfigJSON, &config)
+		if err != nil {
+			return fmt.Errorf("unmarshaling bootstrap config: %v", err)
+		}
+		if config != nil && config.Admin != nil {
+			adminConfig = config.Admin
+		}
+		if cfgEndptSrv != nil {
+			ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+			defer cancel()
+			err := cfgEndptSrv.Shutdown(ctx)
+			if err != nil {
+				return fmt.Errorf("shutting down old admin endpoint: %v", err)
+			}
+		}
+	}
+
+	// extract a singular listener address
+	netw, listenAddrs, err := ParseNetworkAddress(adminConfig.Listen)
+	if err != nil {
+		return fmt.Errorf("parsing admin listener address: %v", err)
+	}
+	if len(listenAddrs) != 1 {
+		return fmt.Errorf("admin endpoint must have exactly one address; cannot listen on %v", listenAddrs)
+	}
+	ln, err := net.Listen(netw, listenAddrs[0])
+	if err != nil {
+		return err
+	}
+
+	mux := http.NewServeMux()
+	mux.HandleFunc("/load", handleLoadConfig)
+	mux.HandleFunc("/stop", handleStop)
+
+	///// BEGIN PPROF STUFF (TODO: Temporary) /////
+	mux.HandleFunc("/debug/pprof/", pprof.Index)
+	mux.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
+	mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
+	mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
+	mux.HandleFunc("/debug/pprof/trace", pprof.Trace)
+	///// END PPROF STUFF //////
+
+	for _, m := range GetModules("admin.routers") {
+		adminrtr := m.New().(AdminRouter)
+		for _, route := range adminrtr.Routes() {
+			mux.Handle(route.Pattern, route)
+		}
+	}
+
+	handler := cors.Default().Handler(mux)
+
+	cfgEndptSrv = &http.Server{
+		Handler:           handler,
+		ReadTimeout:       5 * time.Second,
+		ReadHeaderTimeout: 5 * time.Second,
+		IdleTimeout:       5 * time.Second,
+		MaxHeaderBytes:    1024 * 64,
+	}
+
+	go cfgEndptSrv.Serve(ln)
+
+	log.Println("Caddy 2 admin endpoint listening on", adminConfig.Listen)
+
+	if len(initialConfigJSON) > 0 {
+		err := Load(bytes.NewReader(initialConfigJSON))
+		if err != nil {
+			return fmt.Errorf("loading initial config: %v", err)
+		}
+		log.Println("Caddy 2 serving initial configuration")
+	}
+
+	return nil
+}
+
+// StopAdmin stops the API endpoint.
+func StopAdmin() error {
+	cfgEndptSrvMu.Lock()
+	defer cfgEndptSrvMu.Unlock()
+
+	if cfgEndptSrv == nil {
+		return fmt.Errorf("no server")
+	}
+
+	err := cfgEndptSrv.Shutdown(context.Background()) // TODO
+	if err != nil {
+		return fmt.Errorf("shutting down server: %v", err)
+	}
+
+	cfgEndptSrv = nil
+
+	return nil
+}
+
+// AdminRouter is a type which can return routes for the admin API.
+type AdminRouter interface {
+	Routes() []AdminRoute
+}
+
+// AdminRoute represents a route for the admin endpoint.
+type AdminRoute struct {
+	http.Handler
+	Pattern string
+}
+
+func handleLoadConfig(w http.ResponseWriter, r *http.Request) {
+	if r.Method != http.MethodPost {
+		http.Error(w, http.StatusText(http.StatusMethodNotAllowed), http.StatusMethodNotAllowed)
+		return
+	}
+
+	// if the config is formatted other than Caddy's native
+	// JSON, we need to adapt it before loading it
+	if ctHeader := r.Header.Get("Content-Type"); ctHeader != "" {
+		ct, _, err := mime.ParseMediaType(ctHeader)
+		if err != nil {
+			http.Error(w, "Invalid Content-Type: "+err.Error(), http.StatusBadRequest)
+			return
+		}
+		if !strings.HasSuffix(ct, "/json") {
+			slashIdx := strings.Index(ct, "/")
+			if slashIdx < 0 {
+				http.Error(w, "Malformed Content-Type", http.StatusBadRequest)
+				return
+			}
+			adapterName := ct[slashIdx+1:]
+			cfgAdapter := caddyconfig.GetAdapter(adapterName)
+			if cfgAdapter == nil {
+				http.Error(w, "Unrecognized config adapter: "+adapterName, http.StatusBadRequest)
+				return
+			}
+			body, err := ioutil.ReadAll(http.MaxBytesReader(w, r.Body, 1024*1024))
+			if err != nil {
+				http.Error(w, "Error reading request body: "+err.Error(), http.StatusBadRequest)
+				return
+			}
+			result, warnings, err := cfgAdapter.Adapt(body, nil)
+			if err != nil {
+				log.Printf("[ADMIN][ERROR] adapting config from %s: %v", adapterName, err)
+				http.Error(w, fmt.Sprintf("Adapting config from %s: %v", adapterName, err), http.StatusBadRequest)
+				return
+			}
+			if len(warnings) > 0 {
+				respBody, err := json.Marshal(warnings)
+				if err != nil {
+					log.Printf("[ADMIN][ERROR] marshaling warnings: %v", err)
+				}
+				w.Write(respBody)
+			}
+			// replace original request body with adapted JSON
+			r.Body.Close()
+			r.Body = ioutil.NopCloser(bytes.NewReader(result))
+		}
+	}
+
+	// pass this off to the /config/ endpoint
+	r.URL.Path = "/" + rawConfigKey + "/"
+	handleConfig(w, r)
+}
+
+func handleStop(w http.ResponseWriter, r *http.Request) {
+	if r.Method != http.MethodPost {
+		http.Error(w, http.StatusText(http.StatusMethodNotAllowed), http.StatusMethodNotAllowed)
+		return
+	}
+	log.Println("[ADMIN] Initiating shutdown")
+	if err := stopAndCleanup(); err != nil {
+		log.Printf("[ADMIN][ERROR] stopping: %v \n", err)
+	}
+	log.Println("[ADMIN] Exiting")
+	os.Exit(0)
+}
+
+func stopAndCleanup() error {
+	if err := Stop(); err != nil {
+		return err
+	}
+	certmagic.CleanUpOwnLocks()
+	return nil
+}
+
+// Load loads and starts a configuration.
+func Load(r io.Reader) error {
+	buf := bufPool.Get().(*bytes.Buffer)
+	buf.Reset()
+	defer bufPool.Put(buf)
+
+	_, err := io.Copy(buf, io.LimitReader(r, 1024*1024))
+	if err != nil {
+		return err
+	}
+
+	var cfg *Config
+	err = json.Unmarshal(buf.Bytes(), &cfg)
+	if err != nil {
+		return fmt.Errorf("decoding config: %v", err)
+	}
+
+	err = Run(cfg)
+	if err != nil {
+		return fmt.Errorf("running: %v", err)
+	}
+
+	return nil
+}
+
+// DefaultAdminListen is the address for the admin
+// listener, if none is specified at startup.
+var DefaultAdminListen = "localhost:2019"
+
+var bufPool = sync.Pool{
+	New: func() interface{} {
+		return new(bytes.Buffer)
+	},
+}

admin_test.go

@@ -0,0 +1,44 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddy
+
+import (
+	"strings"
+	"testing"
+)
+
+func BenchmarkLoad(b *testing.B) {
+	for i := 0; i < b.N; i++ {
+		r := strings.NewReader(`{
+			"testval": "Yippee!",
+			"apps": {
+				"http": {
+					"servers": {
+						"myserver": {
+							"listen": ["tcp/localhost:8080-8084"],
+							"read_timeout": "30s"
+						},
+						"yourserver": {
+							"listen": ["127.0.0.1:5000"],
+							"read_header_timeout": "15s"
+						}
+					}
+				}
+			}
+		}
+		`)
+		Load(r)
+	}
+}

app/app.go

@@ -1,76 +0,0 @@
-// Package app holds application-global state to make it accessible
-// by other packages in the application.
-//
-// This package differs from config in that the things in app aren't
-// really related to server configuration.
-package app
-
-import (
-	"errors"
-	"runtime"
-	"strconv"
-	"strings"
-	"sync"
-
-	"github.com/mholt/caddy/server"
-)
-
-const (
-	// Name is the program name
-	Name = "Caddy"
-
-	// Version is the program version
-	Version = "0.7.0"
-)
-
-var (
-	// Servers is a list of all the currently-listening servers
-	Servers []*server.Server
-
-	// ServersMutex protects the Servers slice during changes
-	ServersMutex sync.Mutex
-
-	// Wg is used to wait for all servers to shut down
-	Wg sync.WaitGroup
-
-	// Http2 indicates whether HTTP2 is enabled or not
-	Http2 bool // TODO: temporary flag until http2 is standard
-
-	// Quiet mode hides non-error initialization output
-	Quiet bool
-)
-
-// SetCPU parses string cpu and sets GOMAXPROCS
-// according to its value. It accepts either
-// a number (e.g. 3) or a percent (e.g. 50%).
-func SetCPU(cpu string) error {
-	var numCPU int
-
-	availCPU := runtime.NumCPU()
-
-	if strings.HasSuffix(cpu, "%") {
-		// Percent
-		var percent float32
-		pctStr := cpu[:len(cpu)-1]
-		pctInt, err := strconv.Atoi(pctStr)
-		if err != nil || pctInt < 1 || pctInt > 100 {
-			return errors.New("Invalid CPU value: percentage must be between 1-100")
-		}
-		percent = float32(pctInt) / 100
-		numCPU = int(float32(availCPU) * percent)
-	} else {
-		// Number
-		num, err := strconv.Atoi(cpu)
-		if err != nil || num < 1 {
-			return errors.New("Invalid CPU value: provide a number or percent greater than 0")
-		}
-		numCPU = num
-	}
-
-	if numCPU > availCPU {
-		numCPU = availCPU
-	}
-
-	runtime.GOMAXPROCS(numCPU)
-	return nil
-}

azure-pipelines.yml

@@ -0,0 +1,142 @@
+# Mutilated beyond recognition from the example at:
+# https://docs.microsoft.com/azure/devops/pipelines/languages/go
+
+trigger:
+- v2
+
+strategy:
+  matrix:
+    linux:
+      imageName: ubuntu-16.04
+      gorootDir: /usr/local
+    mac:
+      imageName: macos-10.13
+      gorootDir: /usr/local
+    windows:
+      imageName: windows-2019
+      gorootDir: C:\
+pool:
+  vmImage: $(imageName)
+
+variables:
+  GOROOT: $(gorootDir)/go
+  GOPATH: $(system.defaultWorkingDirectory)/gopath
+  GOBIN:  $(GOPATH)/bin
+  modulePath: '$(GOPATH)/src/github.com/$(build.repository.name)'
+  # TODO: Remove once it's enabled by default
+  GO111MODULE: on
+
+steps:
+- bash: |
+    latestGo=$(curl "https://golang.org/VERSION?m=text")
+    echo "##vso[task.setvariable variable=LATEST_GO]$latestGo"
+    echo "Latest Go version: $latestGo"
+  displayName: "Get latest Go version"
+
+- bash: |
+    sudo rm -f $(which go)
+    echo '##vso[task.prependpath]$(GOBIN)'
+    echo '##vso[task.prependpath]$(GOROOT)/bin'
+    mkdir -p '$(modulePath)'
+    shopt -s extglob
+    shopt -s dotglob
+    mv !(gopath) '$(modulePath)'
+  displayName: Remove old Go, set GOBIN/GOROOT, and move project into GOPATH
+
+# Install Go (this varies by platform)
+- bash: |
+    wget "https://dl.google.com/go/$(LATEST_GO).linux-amd64.tar.gz"
+    sudo tar -C $(gorootDir) -xzf "$(LATEST_GO).linux-amd64.tar.gz"
+  condition: eq( variables['Agent.OS'], 'Linux' )
+  displayName: Install Go on Linux
+
+- bash: |
+    wget "https://dl.google.com/go/$(LATEST_GO).darwin-amd64.tar.gz"
+    sudo tar -C $(gorootDir) -xzf "$(LATEST_GO).darwin-amd64.tar.gz"
+  condition: eq( variables['Agent.OS'], 'Darwin' )
+  displayName: Install Go on macOS
+
+- powershell: |
+    Write-Host "Downloading Go... (please be patient, I am very slow)"
+    (New-Object System.Net.WebClient).DownloadFile("https://dl.google.com/go/$(LATEST_GO).windows-amd64.zip", "$(LATEST_GO).windows-amd64.zip")
+    Write-Host "Extracting Go... (I'm slow too)"
+    Expand-Archive "$(LATEST_GO).windows-amd64.zip" -DestinationPath "$(gorootDir)"
+  condition: eq( variables['Agent.OS'], 'Windows_NT' )
+  displayName: Install Go on Windows
+
+- bash: curl -sfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.19.1
+  displayName: Install golangci-lint
+
+- script: |
+    go get github.com/axw/gocov/gocov
+    go get github.com/AlekSi/gocov-xml
+    go get -u github.com/jstemmer/go-junit-report
+  displayName: Install test and coverage analysis tools
+
+- bash: |
+    printf "Using go at: $(which go)\n"
+    printf "Go version: $(go version)\n"
+    printf "\n\nGo environment:\n\n"
+    go env
+    printf "\n\nSystem environment:\n\n"
+    env
+  displayName: Print Go version and environment
+
+- script: |
+    go get -v -t -d ./...
+    mkdir test-results
+  workingDirectory: '$(modulePath)'
+  displayName: Get dependencies
+
+- script: |
+    (golangci-lint run --out-format junit-xml -E gofmt -E goimports -E misspell) > test-results/lint-result.xml
+    exit 0
+  workingDirectory: '$(modulePath)'
+  continueOnError: true
+  displayName: Run lint check
+
+- script: |
+    (go test -v -coverprofile=cover-profile.out -race ./... 2>&1) > test-results/test-result.out
+  workingDirectory: '$(modulePath)'
+  continueOnError: true
+  displayName: Run tests
+
+- script: |
+    mkdir coverage
+    gocov convert cover-profile.out > coverage/coverage.json
+    # Because Windows doesn't work with input redirection like *nix, but output redirection works.
+    (cat ./coverage/coverage.json | gocov-xml) > coverage/coverage.xml 
+  workingDirectory: '$(modulePath)'
+  displayName: Prepare coverage reports
+
+- script: |
+    (cat ./test-results/test-result.out | go-junit-report) > test-results/test-result.xml
+  workingDirectory: '$(modulePath)'
+  displayName: Prepare test report
+
+- task: PublishCodeCoverageResults@1
+  displayName: Publish test coverage report
+  inputs:
+    codeCoverageTool: Cobertura 
+    summaryFileLocation: $(modulePath)/coverage/coverage.xml
+
+- task: PublishTestResults@2
+  displayName: Publish unit test
+  inputs:
+    testResultsFormat: 'JUnit'
+    testResultsFiles: $(modulePath)/test-results/test-result.xml
+    testRunTitle: $(agent.OS) Unit Test
+    mergeTestResults: false
+
+- task: PublishTestResults@2
+  displayName: Publish lint results
+  inputs:
+    testResultsFormat: 'JUnit'
+    testResultsFiles: $(modulePath)/test-results/lint-result.xml
+    testRunTitle: $(agent.OS) Lint
+    mergeTestResults: false
+
+- bash: |
+    exit 1
+  condition: eq(variables['Agent.JobStatus'], 'SucceededWithIssues')
+  displayName: Coerce correct build result

caddy.go

@@ -0,0 +1,272 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddy
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"log"
+	"runtime/debug"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/mholt/certmagic"
+)
+
+// Config represents a Caddy configuration.
+type Config struct {
+	Admin *AdminConfig `json:"admin,omitempty"`
+
+	StorageRaw json.RawMessage `json:"storage,omitempty"`
+	storage    certmagic.Storage
+
+	AppsRaw map[string]json.RawMessage `json:"apps,omitempty"`
+
+	// apps stores the decoded Apps values,
+	// keyed by module name.
+	apps map[string]App
+
+	cancelFunc context.CancelFunc
+}
+
+// App is a thing that Caddy runs.
+type App interface {
+	Start() error
+	Stop() error
+}
+
+// Run runs Caddy with the given config.
+func Run(newCfg *Config) error {
+	currentCfgMu.Lock()
+	defer currentCfgMu.Unlock()
+
+	// run the new config and start all its apps
+	err := run(newCfg, true)
+	if err != nil {
+		return err
+	}
+
+	// swap old config with the new one
+	oldCfg := currentCfg
+	currentCfg = newCfg
+
+	// Stop, Cleanup each old app
+	unsyncedStop(oldCfg)
+
+	return nil
+}
+
+// run runs newCfg and starts all its apps if
+// start is true. If any errors happen, cleanup
+// is performed if any modules were provisioned;
+// apps that were started already will be stopped,
+// so this function should not leak resources if
+// an error is returned.
+func run(newCfg *Config, start bool) error {
+	if newCfg == nil {
+		return nil
+	}
+
+	// because we will need to roll back any state
+	// modifications if this function errors, we
+	// keep a single error value and scope all
+	// sub-operations to their own functions to
+	// ensure this error value does not get
+	// overridden or missed when it should have
+	// been set by a short assignment
+	var err error
+
+	// prepare the new config for use
+	newCfg.apps = make(map[string]App)
+
+	// create a context within which to load
+	// modules - essentially our new config's
+	// execution environment; be sure that
+	// cleanup occurs when we return if there
+	// was an error; if no error, it will get
+	// cleaned up on next config cycle
+	ctx, cancel := NewContext(Context{Context: context.Background(), cfg: newCfg})
+	defer func() {
+		if err != nil {
+			cancel() // clean up now
+		}
+	}()
+	newCfg.cancelFunc = cancel // clean up later
+
+	// set up storage and make it CertMagic's default storage, too
+	err = func() error {
+		if newCfg.StorageRaw != nil {
+			val, err := ctx.LoadModuleInline("module", "caddy.storage", newCfg.StorageRaw)
+			if err != nil {
+				return fmt.Errorf("loading storage module: %v", err)
+			}
+			stor, err := val.(StorageConverter).CertMagicStorage()
+			if err != nil {
+				return fmt.Errorf("creating storage value: %v", err)
+			}
+			newCfg.storage = stor
+			newCfg.StorageRaw = nil // allow GC to deallocate
+		}
+		if newCfg.storage == nil {
+			newCfg.storage = &certmagic.FileStorage{Path: dataDir()}
+		}
+		certmagic.Default.Storage = newCfg.storage
+
+		return nil
+	}()
+	if err != nil {
+		return err
+	}
+
+	// Load, Provision, Validate each app and their submodules
+	err = func() error {
+		for modName, rawMsg := range newCfg.AppsRaw {
+			val, err := ctx.LoadModule(modName, rawMsg)
+			if err != nil {
+				return fmt.Errorf("loading app module '%s': %v", modName, err)
+			}
+			newCfg.apps[modName] = val.(App)
+		}
+		return nil
+	}()
+	if err != nil {
+		return err
+	}
+
+	if !start {
+		return nil
+	}
+
+	// Start
+	return func() error {
+		var started []string
+		for name, a := range newCfg.apps {
+			err := a.Start()
+			if err != nil {
+				// an app failed to start, so we need to stop
+				// all other apps that were already started
+				for _, otherAppName := range started {
+					err2 := newCfg.apps[otherAppName].Stop()
+					if err2 != nil {
+						err = fmt.Errorf("%v; additionally, aborting app %s: %v",
+							err, otherAppName, err2)
+					}
+				}
+				return fmt.Errorf("%s app module: start: %v", name, err)
+			}
+			started = append(started, name)
+		}
+		return nil
+	}()
+}
+
+// Stop stops running the current configuration.
+// It is the antithesis of Run(). This function
+// will log any errors that occur during the
+// stopping of individual apps and continue to
+// stop the others.
+func Stop() error {
+	currentCfgMu.Lock()
+	defer currentCfgMu.Unlock()
+	unsyncedStop(currentCfg)
+	currentCfg = nil
+	return nil
+}
+
+// unsyncedStop stops cfg from running, but if
+// applicable, you need to acquire locks yourself.
+// It is a no-op if cfg is nil. If any app
+// returns an error when stopping, it is logged
+// and the function continues with the next app.
+// This function assumes all apps in cfg were
+// successfully started.
+func unsyncedStop(cfg *Config) {
+	if cfg == nil {
+		return
+	}
+
+	// stop each app
+	for name, a := range cfg.apps {
+		err := a.Stop()
+		if err != nil {
+			log.Printf("[ERROR] stop %s: %v", name, err)
+		}
+	}
+
+	// clean up all modules
+	cfg.cancelFunc()
+}
+
+// Validate loads, provisions, and validates
+// cfg, but does not start running it.
+func Validate(cfg *Config) error {
+	return run(cfg, false)
+}
+
+// Duration is a JSON-string-unmarshable duration type.
+type Duration time.Duration
+
+// UnmarshalJSON satisfies json.Unmarshaler.
+func (d *Duration) UnmarshalJSON(b []byte) error {
+	dd, err := time.ParseDuration(strings.Trim(string(b), `"`))
+	if err != nil {
+		return err
+	}
+	*d = Duration(dd)
+	return nil
+}
+
+// GoModule returns the build info of this Caddy
+// build from debug.BuildInfo (requires Go modules).
+// If no version information is available, a non-nil
+// value will still be returned, but with an
+// unknown version.
+func GoModule() *debug.Module {
+	var mod debug.Module
+	return goModule(&mod)
+}
+
+// goModule holds the actual implementation of GoModule.
+// Allocating debug.Module in GoModule() and passing a
+// reference to goModule enables mid-stack inlining.
+func goModule(mod *debug.Module) *debug.Module {
+	mod.Version = "unknown"
+	bi, ok := debug.ReadBuildInfo()
+	if ok {
+		mod.Path = bi.Main.Path
+		// The recommended way to build Caddy involves
+		// creating a separate main module, which
+		// TODO: track related Go issue: https://github.com/golang/go/issues/29228
+		// once that issue is fixed, we should just be able to use bi.Main... hopefully.
+		for _, dep := range bi.Deps {
+			if dep.Path == "github.com/caddyserver/caddy/v2" {
+				return dep
+			}
+		}
+		return &bi.Main
+	}
+	return mod
+}
+
+// CtxKey is a value type for use with context.WithValue.
+type CtxKey string
+
+// currentCfg is the currently-loaded configuration.
+var (
+	currentCfg   *Config
+	currentCfgMu sync.RWMutex
+)

caddyconfig/caddyfile/adapter.go

@@ -0,0 +1,87 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddyfile
+
+import (
+	"bytes"
+	"encoding/json"
+	"fmt"
+
+	"github.com/caddyserver/caddy/v2"
+	"github.com/caddyserver/caddy/v2/caddyconfig"
+)
+
+// Adapter adapts Caddyfile to Caddy JSON.
+type Adapter struct {
+	ServerType ServerType
+}
+
+// Adapt converts the Caddyfile config in body to Caddy JSON.
+func (a Adapter) Adapt(body []byte, options map[string]interface{}) ([]byte, []caddyconfig.Warning, error) {
+	if a.ServerType == nil {
+		return nil, nil, fmt.Errorf("no server type")
+	}
+	if options == nil {
+		options = make(map[string]interface{})
+	}
+
+	filename, _ := options["filename"].(string)
+	if filename == "" {
+		filename = "Caddyfile"
+	}
+
+	serverBlocks, err := Parse(filename, bytes.NewReader(body))
+	if err != nil {
+		return nil, nil, err
+	}
+
+	cfg, warnings, err := a.ServerType.Setup(serverBlocks, options)
+	if err != nil {
+		return nil, warnings, err
+	}
+
+	marshalFunc := json.Marshal
+	if options["pretty"] == "true" {
+		marshalFunc = caddyconfig.JSONIndent
+	}
+	result, err := marshalFunc(cfg)
+
+	return result, warnings, err
+}
+
+// Unmarshaler is a type that can unmarshal
+// Caddyfile tokens to set itself up for a
+// JSON encoding. The goal of an unmarshaler
+// is not to set itself up for actual use,
+// but to set itself up for being marshaled
+// into JSON. Caddyfile-unmarshaled values
+// will not be used directly; they will be
+// encoded as JSON and then used from that.
+type Unmarshaler interface {
+	UnmarshalCaddyfile(d *Dispenser) error
+}
+
+// ServerType is a type that can evaluate a Caddyfile and set up a caddy config.
+type ServerType interface {
+	// Setup takes the server blocks which
+	// contain tokens, as well as options
+	// (e.g. CLI flags) and creates a Caddy
+	// config, along with any warnings or
+	// an error.
+	Setup([]ServerBlock, map[string]interface{}) (*caddy.Config, []caddyconfig.Warning, error)
+}
+
+// Interface guard
+var _ caddyconfig.Adapter = (*Adapter)(nil)

caddyconfig/caddyfile/dispenser.go

@@ -0,0 +1,371 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddyfile
+
+import (
+	"errors"
+	"fmt"
+	"strings"
+)
+
+// Dispenser is a type that dispenses tokens, similarly to a lexer,
+// except that it can do so with some notion of structure. An empty
+// Dispenser is invalid; call NewDispenser to make a proper instance.
+type Dispenser struct {
+	tokens  []Token
+	cursor  int
+	nesting int
+}
+
+// NewDispenser returns a Dispenser filled with the given tokens.
+func NewDispenser(tokens []Token) *Dispenser {
+	return &Dispenser{
+		tokens: tokens,
+		cursor: -1,
+	}
+}
+
+// Next loads the next token. Returns true if a token
+// was loaded; false otherwise. If false, all tokens
+// have been consumed.
+func (d *Dispenser) Next() bool {
+	if d.cursor < len(d.tokens)-1 {
+		d.cursor++
+		return true
+	}
+	return false
+}
+
+// Prev moves to the previous token. It does the inverse
+// of Next(), except this function may decrement the cursor
+// to -1 so that the next call to Next() points to the
+// first token; this allows dispensing to "start over". This
+// method returns true if the cursor ends up pointing to a
+// valid token.
+func (d *Dispenser) Prev() bool {
+	if d.cursor > -1 {
+		d.cursor--
+		return d.cursor > -1
+	}
+	return false
+}
+
+// NextArg loads the next token if it is on the same
+// line and if it is not a block opening (open curly
+// brace). Returns true if an argument token was
+// loaded; false otherwise. If false, all tokens on
+// the line have been consumed except for potentially
+// a block opening. It handles imported tokens
+// correctly.
+func (d *Dispenser) NextArg() bool {
+	if !d.nextOnSameLine() {
+		return false
+	}
+	if d.Val() == "{" {
+		// roll back; a block opening is not an argument
+		d.cursor--
+		return false
+	}
+	return true
+}
+
+// nextOnSameLine advances the cursor if the next
+// token is on the same line of the same file.
+func (d *Dispenser) nextOnSameLine() bool {
+	if d.cursor < 0 {
+		d.cursor++
+		return true
+	}
+	if d.cursor >= len(d.tokens) {
+		return false
+	}
+	if d.cursor < len(d.tokens)-1 &&
+		d.tokens[d.cursor].File == d.tokens[d.cursor+1].File &&
+		d.tokens[d.cursor].Line+d.numLineBreaks(d.cursor) == d.tokens[d.cursor+1].Line {
+		d.cursor++
+		return true
+	}
+	return false
+}
+
+// NextLine loads the next token only if it is not on the same
+// line as the current token, and returns true if a token was
+// loaded; false otherwise. If false, there is not another token
+// or it is on the same line. It handles imported tokens correctly.
+func (d *Dispenser) NextLine() bool {
+	if d.cursor < 0 {
+		d.cursor++
+		return true
+	}
+	if d.cursor >= len(d.tokens) {
+		return false
+	}
+	if d.cursor < len(d.tokens)-1 &&
+		(d.tokens[d.cursor].File != d.tokens[d.cursor+1].File ||
+			d.tokens[d.cursor].Line+d.numLineBreaks(d.cursor) < d.tokens[d.cursor+1].Line) {
+		d.cursor++
+		return true
+	}
+	return false
+}
+
+// NextBlock can be used as the condition of a for loop
+// to load the next token as long as it opens a block or
+// is already in a block nested more than initialNestingLevel.
+// In other words, a loop over NextBlock() will iterate
+// all tokens in the block assuming the next token is an
+// open curly brace, until the matching closing brace.
+// The open and closing brace tokens for the outer-most
+// block will be consumed internally and omitted from
+// the iteration.
+//
+// Proper use of this method looks like this:
+//
+//     for nesting := d.Nesting(); d.NextBlock(nesting); {
+//     }
+//
+// However, in simple cases where it is known that the
+// Dispenser is new and has not already traversed state
+// by a loop over NextBlock(), this will do:
+//
+//     for d.NextBlock(0) {
+//     }
+//
+// As with other token parsing logic, a loop over
+// NextBlock() should be contained within a loop over
+// Next(), as it is usually prudent to skip the initial
+// token.
+func (d *Dispenser) NextBlock(initialNestingLevel int) bool {
+	if d.nesting > initialNestingLevel {
+		if !d.Next() {
+			return false // should be EOF error
+		}
+		if d.Val() == "}" {
+			d.nesting--
+		}
+		return d.nesting > initialNestingLevel
+	}
+	if !d.nextOnSameLine() { // block must open on same line
+		return false
+	}
+	if d.Val() != "{" {
+		d.cursor-- // roll back if not opening brace
+		return false
+	}
+	d.Next() // consume open curly brace
+	if d.Val() == "}" {
+		return false // open and then closed right away
+	}
+	d.nesting++
+	return true
+}
+
+// Nesting returns the current nesting level. Necessary
+// if using NextBlock()
+func (d *Dispenser) Nesting() int {
+	return d.nesting
+}
+
+// Val gets the text of the current token. If there is no token
+// loaded, it returns empty string.
+func (d *Dispenser) Val() string {
+	if d.cursor < 0 || d.cursor >= len(d.tokens) {
+		return ""
+	}
+	return d.tokens[d.cursor].Text
+}
+
+// Line gets the line number of the current token.
+// If there is no token loaded, it returns 0.
+func (d *Dispenser) Line() int {
+	if d.cursor < 0 || d.cursor >= len(d.tokens) {
+		return 0
+	}
+	return d.tokens[d.cursor].Line
+}
+
+// File gets the filename where the current token originated.
+func (d *Dispenser) File() string {
+	if d.cursor < 0 || d.cursor >= len(d.tokens) {
+		return ""
+	}
+	return d.tokens[d.cursor].File
+}
+
+// Args is a convenience function that loads the next arguments
+// (tokens on the same line) into an arbitrary number of strings
+// pointed to in targets. If there are not enough argument tokens
+// available to fill targets, false is returned and the remaining
+// targets are left unchanged. If all the targets are filled,
+// then true is returned.
+func (d *Dispenser) Args(targets ...*string) bool {
+	for i := 0; i < len(targets); i++ {
+		if !d.NextArg() {
+			return false
+		}
+		*targets[i] = d.Val()
+	}
+	return true
+}
+
+// AllArgs is like Args, but if there are more argument tokens
+// available than there are targets, false is returned. The
+// number of available argument tokens must match the number of
+// targets exactly to return true.
+func (d *Dispenser) AllArgs(targets ...*string) bool {
+	if !d.Args(targets...) {
+		return false
+	}
+	if d.NextArg() {
+		d.Prev()
+		return false
+	}
+	return true
+}
+
+// RemainingArgs loads any more arguments (tokens on the same line)
+// into a slice and returns them. Open curly brace tokens also indicate
+// the end of arguments, and the curly brace is not included in
+// the return value nor is it loaded.
+func (d *Dispenser) RemainingArgs() []string {
+	var args []string
+	for d.NextArg() {
+		args = append(args, d.Val())
+	}
+	return args
+}
+
+// NewFromNextTokens returns a new dispenser with a copy of
+// the tokens from the current token until the end of the
+// "directive" whether that be to the end of the line or
+// the end of a block that starts at the end of the line;
+// in other words, until the end of the segment.
+func (d *Dispenser) NewFromNextTokens() *Dispenser {
+	tkns := []Token{d.Token()}
+	for d.NextArg() {
+		tkns = append(tkns, d.Token())
+	}
+	var openedBlock bool
+	for nesting := d.Nesting(); d.NextBlock(nesting); {
+		if !openedBlock {
+			// because NextBlock() consumes the initial open
+			// curly brace, we rewind here to append it, since
+			// our case is special in that we want to include
+			// all the tokens including surrounding curly braces
+			// for a new dispenser to have
+			d.Prev()
+			tkns = append(tkns, d.Token())
+			d.Next()
+			openedBlock = true
+		}
+		tkns = append(tkns, d.Token())
+	}
+	if openedBlock {
+		// include closing brace accordingly
+		tkns = append(tkns, d.Token())
+	}
+	return NewDispenser(tkns)
+}
+
+// Token returns the current token.
+func (d *Dispenser) Token() Token {
+	if d.cursor < 0 || d.cursor >= len(d.tokens) {
+		return Token{}
+	}
+	return d.tokens[d.cursor]
+}
+
+// Reset sets d's cursor to the beginning, as
+// if this was a new and unused dispenser.
+func (d *Dispenser) Reset() {
+	d.cursor = -1
+	d.nesting = 0
+}
+
+// ArgErr returns an argument error, meaning that another
+// argument was expected but not found. In other words,
+// a line break or open curly brace was encountered instead of
+// an argument.
+func (d *Dispenser) ArgErr() error {
+	if d.Val() == "{" {
+		return d.Err("Unexpected token '{', expecting argument")
+	}
+	return d.Errf("Wrong argument count or unexpected line ending after '%s'", d.Val())
+}
+
+// SyntaxErr creates a generic syntax error which explains what was
+// found and what was expected.
+func (d *Dispenser) SyntaxErr(expected string) error {
+	msg := fmt.Sprintf("%s:%d - Syntax error: Unexpected token '%s', expecting '%s'", d.File(), d.Line(), d.Val(), expected)
+	return errors.New(msg)
+}
+
+// EOFErr returns an error indicating that the dispenser reached
+// the end of the input when searching for the next token.
+func (d *Dispenser) EOFErr() error {
+	return d.Errf("Unexpected EOF")
+}
+
+// Err generates a custom parse-time error with a message of msg.
+func (d *Dispenser) Err(msg string) error {
+	msg = fmt.Sprintf("%s:%d - Error during parsing: %s", d.File(), d.Line(), msg)
+	return errors.New(msg)
+}
+
+// Errf is like Err, but for formatted error messages
+func (d *Dispenser) Errf(format string, args ...interface{}) error {
+	return d.Err(fmt.Sprintf(format, args...))
+}
+
+// Delete deletes the current token and returns the updated slice
+// of tokens. The cursor is not advanced to the next token.
+// Because deletion modifies the underlying slice, this method
+// should only be called if you have access to the original slice
+// of tokens and/or are using the slice of tokens outside this
+// Dispenser instance. If you do not re-assign the slice with the
+// return value of this method, inconsistencies in the token
+// array will become apparent (or worse, hide from you like they
+// did me for 3 and a half freaking hours late one night).
+func (d *Dispenser) Delete() []Token {
+	if d.cursor >= 0 && d.cursor < len(d.tokens)-1 {
+		d.tokens = append(d.tokens[:d.cursor], d.tokens[d.cursor+1:]...)
+		d.cursor--
+	}
+	return d.tokens
+}
+
+// numLineBreaks counts how many line breaks are in the token
+// value given by the token index tknIdx. It returns 0 if the
+// token does not exist or there are no line breaks.
+func (d *Dispenser) numLineBreaks(tknIdx int) int {
+	if tknIdx < 0 || tknIdx >= len(d.tokens) {
+		return 0
+	}
+	return strings.Count(d.tokens[tknIdx].Text, "\n")
+}
+
+// isNewLine determines whether the current token is on a different
+// line (higher line number) than the previous token. It handles imported
+// tokens correctly. If there isn't a previous token, it returns true.
+func (d *Dispenser) isNewLine() bool {
+	if d.cursor < 1 {
+		return true
+	}
+	if d.cursor > len(d.tokens)-1 {
+		return false
+	}
+	return d.tokens[d.cursor-1].File != d.tokens[d.cursor].File ||
+		d.tokens[d.cursor-1].Line+d.numLineBreaks(d.cursor-1) < d.tokens[d.cursor].Line
+}

caddyconfig/caddyfile/dispenser_test.go

@@ -1,6 +1,22 @@
-package parse
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddyfile
 
 import (
+	"io"
+	"log"
 	"reflect"
 	"strings"
 	"testing"
@@ -11,7 +27,7 @@ func TestDispenser_Val_Next(t *testing.T) {
 			  dir1 arg1
 			  dir2 arg2 arg3
 			  dir3`
-	d := NewDispenser("Testfile", strings.NewReader(input))
+	d := newTestDispenser(input)
 
 	if val := d.Val(); val != "" {
 		t.Fatalf("Val(): Should return empty string when no token loaded; got '%s'", val)
@@ -49,7 +65,7 @@ func TestDispenser_NextArg(t *testing.T) {
 	input := `dir1 arg1
 			  dir2 arg2 arg3
 			  dir3`
-	d := NewDispenser("Testfile", strings.NewReader(input))
+	d := newTestDispenser(input)
 
 	assertNext := func(shouldLoad bool, expectedVal string, expectedCursor int) {
 		if d.Next() != shouldLoad {
@@ -64,7 +80,7 @@ func TestDispenser_NextArg(t *testing.T) {
 	}
 
 	assertNextArg := func(expectedVal string, loadAnother bool, expectedCursor int) {
-		if d.NextArg() != true {
+		if !d.NextArg() {
 			t.Error("NextArg(): Should load next argument but got false instead")
 		}
 		if d.cursor != expectedCursor {
@@ -74,7 +90,7 @@ func TestDispenser_NextArg(t *testing.T) {
 			t.Errorf("Val(): Expected '%s' but got '%s'", expectedVal, val)
 		}
 		if !loadAnother {
-			if d.NextArg() != false {
+			if d.NextArg() {
 				t.Fatalf("NextArg(): Should NOT load another argument, but got true instead (val: '%s')", d.Val())
 			}
 			if d.cursor != expectedCursor {
@@ -96,7 +112,7 @@ func TestDispenser_NextLine(t *testing.T) {
 	input := `host:port
 			  dir1 arg1
 			  dir2 arg2 arg3`
-	d := NewDispenser("Testfile", strings.NewReader(input))
+	d := newTestDispenser(input)
 
 	assertNextLine := func(shouldLoad bool, expectedVal string, expectedCursor int) {
 		if d.NextLine() != shouldLoad {
@@ -129,10 +145,10 @@ func TestDispenser_NextBlock(t *testing.T) {
 			  }
 			  foobar2 {
 			  }`
-	d := NewDispenser("Testfile", strings.NewReader(input))
+	d := newTestDispenser(input)
 
 	assertNextBlock := func(shouldLoad bool, expectedCursor, expectedNesting int) {
-		if loaded := d.NextBlock(); loaded != shouldLoad {
+		if loaded := d.NextBlock(0); loaded != shouldLoad {
 			t.Errorf("NextBlock(): Should return %v but got %v", shouldLoad, loaded)
 		}
 		if d.cursor != expectedCursor {
@@ -159,7 +175,7 @@ func TestDispenser_Args(t *testing.T) {
 			  dir2 arg4 arg5
 			  dir3 arg6 arg7
 			  dir4`
-	d := NewDispenser("Testfile", strings.NewReader(input))
+	d := newTestDispenser(input)
 
 	d.Next() // dir1
 
@@ -226,7 +242,7 @@ func TestDispenser_RemainingArgs(t *testing.T) {
 			  dir2 arg4 arg5
 			  dir3 arg6 { arg7
 			  dir4`
-	d := NewDispenser("Testfile", strings.NewReader(input))
+	d := newTestDispenser(input)
 
 	d.Next() // dir1
 
@@ -263,7 +279,7 @@ func TestDispenser_ArgErr_Err(t *testing.T) {
 	input := `dir1 {
 			  }
 			  dir2 arg1 arg2`
-	d := NewDispenser("Testfile", strings.NewReader(input))
+	d := newTestDispenser(input)
 
 	d.cursor = 1 // {
 
@@ -290,3 +306,11 @@ func TestDispenser_ArgErr_Err(t *testing.T) {
 		t.Errorf("Expected error message with custom message in it ('foobar'); got '%v'", err)
 	}
 }
+
+func newTestDispenser(input string) *Dispenser {
+	tokens, err := allTokens("Testfile", strings.NewReader(input))
+	if err != nil && err != io.EOF {
+		log.Fatalf("getting all tokens from input: %v", err)
+	}
+	return NewDispenser(tokens)
+}

caddyconfig/caddyfile/lexer.go

@@ -1,4 +1,18 @@
-package parse
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddyfile
 
 import (
 	"bufio"
@@ -12,22 +26,38 @@ type (
 	// are separated by whitespace. A word can be enclosed
 	// in quotes if it contains whitespace.
 	lexer struct {
-		reader *bufio.Reader
-		token  token
-		line   int
+		reader       *bufio.Reader
+		token        Token
+		line         int
+		skippedLines int
 	}
 
-	// token represents a single parsable unit.
-	token struct {
-		line int
-		text string
+	// Token represents a single parsable unit.
+	Token struct {
+		File string
+		Line int
+		Text string
 	}
 )
 
 // load prepares the lexer to scan an input for tokens.
+// It discards any leading byte order mark.
 func (l *lexer) load(input io.Reader) error {
 	l.reader = bufio.NewReader(input)
 	l.line = 1
+
+	// discard byte order mark, if present
+	firstCh, _, err := l.reader.ReadRune()
+	if err != nil {
+		return err
+	}
+	if firstCh != 0xFEFF {
+		err := l.reader.UnreadRune()
+		if err != nil {
+			return err
+		}
+	}
+
 	return nil
 }
 
@@ -46,7 +76,7 @@ func (l *lexer) next() bool {
 	var comment, quoted, escaped bool
 
 	makeToken := func() bool {
-		l.token.text = string(val)
+		l.token.Text = string(val)
 		return true
 	}
 
@@ -62,27 +92,30 @@ func (l *lexer) next() bool {
 			panic(err)
 		}
 
+		if !escaped && ch == '\\' {
+			escaped = true
+			continue
+		}
+
 		if quoted {
-			if !escaped {
-				if ch == '\\' {
-					escaped = true
-					continue
-				} else if ch == '"' {
+			if escaped {
+				// all is literal in quoted area,
+				// so only escape quotes
+				if ch != '"' {
+					val = append(val, '\\')
+				}
+				escaped = false
+			} else {
+				if ch == '"' {
 					quoted = false
 					return makeToken()
 				}
 			}
 			if ch == '\n' {
-				l.line++
-			}
-			if escaped {
-				// only escape quotes
-				if ch != '"' {
-					val = append(val, '\\')
-				}
+				l.line += 1 + l.skippedLines
+				l.skippedLines = 0
 			}
 			val = append(val, ch)
-			escaped = false
 			continue
 		}
 
@@ -91,7 +124,13 @@ func (l *lexer) next() bool {
 				continue
 			}
 			if ch == '\n' {
-				l.line++
+				if escaped {
+					l.skippedLines++
+					escaped = false
+				} else {
+					l.line += 1 + l.skippedLines
+					l.skippedLines = 0
+				}
 				comment = false
 			}
 			if len(val) > 0 {
@@ -103,13 +142,12 @@ func (l *lexer) next() bool {
 		if ch == '#' {
 			comment = true
 		}
-
 		if comment {
 			continue
 		}
 
 		if len(val) == 0 {
-			l.token = token{line: l.line}
+			l.token = Token{Line: l.line}
 			if ch == '"' {
 				quoted = true
 				continue

caddyconfig/caddyfile/lexer_test.go

@@ -0,0 +1,232 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddyfile
+
+import (
+	"log"
+	"strings"
+	"testing"
+)
+
+type lexerTestCase struct {
+	input    string
+	expected []Token
+}
+
+func TestLexer(t *testing.T) {
+	testCases := []lexerTestCase{
+		{
+			input: `host:123`,
+			expected: []Token{
+				{Line: 1, Text: "host:123"},
+			},
+		},
+		{
+			input: `host:123
+
+					directive`,
+			expected: []Token{
+				{Line: 1, Text: "host:123"},
+				{Line: 3, Text: "directive"},
+			},
+		},
+		{
+			input: `host:123 {
+						directive
+					}`,
+			expected: []Token{
+				{Line: 1, Text: "host:123"},
+				{Line: 1, Text: "{"},
+				{Line: 2, Text: "directive"},
+				{Line: 3, Text: "}"},
+			},
+		},
+		{
+			input: `host:123 { directive }`,
+			expected: []Token{
+				{Line: 1, Text: "host:123"},
+				{Line: 1, Text: "{"},
+				{Line: 1, Text: "directive"},
+				{Line: 1, Text: "}"},
+			},
+		},
+		{
+			input: `host:123 {
+						#comment
+						directive
+						# comment
+						foobar # another comment
+					}`,
+			expected: []Token{
+				{Line: 1, Text: "host:123"},
+				{Line: 1, Text: "{"},
+				{Line: 3, Text: "directive"},
+				{Line: 5, Text: "foobar"},
+				{Line: 6, Text: "}"},
+			},
+		},
+		{
+			input: `a "quoted value" b
+					foobar`,
+			expected: []Token{
+				{Line: 1, Text: "a"},
+				{Line: 1, Text: "quoted value"},
+				{Line: 1, Text: "b"},
+				{Line: 2, Text: "foobar"},
+			},
+		},
+		{
+			input: `A "quoted \"value\" inside" B`,
+			expected: []Token{
+				{Line: 1, Text: "A"},
+				{Line: 1, Text: `quoted "value" inside`},
+				{Line: 1, Text: "B"},
+			},
+		},
+		{
+			input: "An escaped \"newline\\\ninside\" quotes",
+			expected: []Token{
+				{Line: 1, Text: "An"},
+				{Line: 1, Text: "escaped"},
+				{Line: 1, Text: "newline\\\ninside"},
+				{Line: 2, Text: "quotes"},
+			},
+		},
+		{
+			input: "An escaped newline\\\noutside quotes",
+			expected: []Token{
+				{Line: 1, Text: "An"},
+				{Line: 1, Text: "escaped"},
+				{Line: 1, Text: "newline"},
+				{Line: 1, Text: "outside"},
+				{Line: 1, Text: "quotes"},
+			},
+		},
+		{
+			input: "line1\\\nescaped\nline2\nline3",
+			expected: []Token{
+				{Line: 1, Text: "line1"},
+				{Line: 1, Text: "escaped"},
+				{Line: 3, Text: "line2"},
+				{Line: 4, Text: "line3"},
+			},
+		},
+		{
+			input: "line1\\\nescaped1\\\nescaped2\nline4\nline5",
+			expected: []Token{
+				{Line: 1, Text: "line1"},
+				{Line: 1, Text: "escaped1"},
+				{Line: 1, Text: "escaped2"},
+				{Line: 4, Text: "line4"},
+				{Line: 5, Text: "line5"},
+			},
+		},
+		{
+			input: `"unescapable\ in quotes"`,
+			expected: []Token{
+				{Line: 1, Text: `unescapable\ in quotes`},
+			},
+		},
+		{
+			input: `"don't\escape"`,
+			expected: []Token{
+				{Line: 1, Text: `don't\escape`},
+			},
+		},
+		{
+			input: `"don't\\escape"`,
+			expected: []Token{
+				{Line: 1, Text: `don't\\escape`},
+			},
+		},
+		{
+			input: `A "quoted value with line
+					break inside" {
+						foobar
+					}`,
+			expected: []Token{
+				{Line: 1, Text: "A"},
+				{Line: 1, Text: "quoted value with line\n\t\t\t\t\tbreak inside"},
+				{Line: 2, Text: "{"},
+				{Line: 3, Text: "foobar"},
+				{Line: 4, Text: "}"},
+			},
+		},
+		{
+			input: `"C:\php\php-cgi.exe"`,
+			expected: []Token{
+				{Line: 1, Text: `C:\php\php-cgi.exe`},
+			},
+		},
+		{
+			input: `empty "" string`,
+			expected: []Token{
+				{Line: 1, Text: `empty`},
+				{Line: 1, Text: ``},
+				{Line: 1, Text: `string`},
+			},
+		},
+		{
+			input: "skip those\r\nCR characters",
+			expected: []Token{
+				{Line: 1, Text: "skip"},
+				{Line: 1, Text: "those"},
+				{Line: 2, Text: "CR"},
+				{Line: 2, Text: "characters"},
+			},
+		},
+		{
+			input: "\xEF\xBB\xBF:8080", // test with leading byte order mark
+			expected: []Token{
+				{Line: 1, Text: ":8080"},
+			},
+		},
+	}
+
+	for i, testCase := range testCases {
+		actual := tokenize(testCase.input)
+		lexerCompare(t, i, testCase.expected, actual)
+	}
+}
+
+func tokenize(input string) (tokens []Token) {
+	l := lexer{}
+	if err := l.load(strings.NewReader(input)); err != nil {
+		log.Printf("[ERROR] load failed: %v", err)
+	}
+	for l.next() {
+		tokens = append(tokens, l.token)
+	}
+	return
+}
+
+func lexerCompare(t *testing.T, n int, expected, actual []Token) {
+	if len(expected) != len(actual) {
+		t.Errorf("Test case %d: expected %d token(s) but got %d", n, len(expected), len(actual))
+	}
+
+	for i := 0; i < len(actual) && i < len(expected); i++ {
+		if actual[i].Line != expected[i].Line {
+			t.Errorf("Test case %d token %d ('%s'): expected line %d but was line %d",
+				n, i, expected[i].Text, expected[i].Line, actual[i].Line)
+			break
+		}
+		if actual[i].Text != expected[i].Text {
+			t.Errorf("Test case %d token %d: expected text '%s' but was '%s'",
+				n, i, expected[i].Text, actual[i].Text)
+			break
+		}
+	}
+}

caddyconfig/caddyfile/parse.go

@@ -0,0 +1,516 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddyfile
+
+import (
+	"io"
+	"log"
+	"os"
+	"path/filepath"
+	"strings"
+)
+
+// Parse parses the input just enough to group tokens, in
+// order, by server block. No further parsing is performed.
+// Server blocks are returned in the order in which they appear.
+// Directives that do not appear in validDirectives will cause
+// an error. If you do not want to check for valid directives,
+// pass in nil instead.
+func Parse(filename string, input io.Reader) ([]ServerBlock, error) {
+	tokens, err := allTokens(filename, input)
+	if err != nil {
+		return nil, err
+	}
+	p := parser{Dispenser: NewDispenser(tokens)}
+	return p.parseAll()
+}
+
+// allTokens lexes the entire input, but does not parse it.
+// It returns all the tokens from the input, unstructured
+// and in order.
+func allTokens(filename string, input io.Reader) ([]Token, error) {
+	l := new(lexer)
+	err := l.load(input)
+	if err != nil {
+		return nil, err
+	}
+	var tokens []Token
+	for l.next() {
+		l.token.File = filename
+		tokens = append(tokens, l.token)
+	}
+	return tokens, nil
+}
+
+type parser struct {
+	*Dispenser
+	block           ServerBlock // current server block being parsed
+	eof             bool        // if we encounter a valid EOF in a hard place
+	definedSnippets map[string][]Token
+	nesting         int
+}
+
+func (p *parser) parseAll() ([]ServerBlock, error) {
+	var blocks []ServerBlock
+
+	for p.Next() {
+		err := p.parseOne()
+		if err != nil {
+			return blocks, err
+		}
+		if len(p.block.Keys) > 0 || len(p.block.Segments) > 0 {
+			blocks = append(blocks, p.block)
+		}
+		if p.nesting > 0 {
+			return blocks, p.EOFErr()
+		}
+	}
+
+	return blocks, nil
+}
+
+func (p *parser) parseOne() error {
+	p.block = ServerBlock{}
+	return p.begin()
+}
+
+func (p *parser) begin() error {
+	if len(p.tokens) == 0 {
+		return nil
+	}
+
+	err := p.addresses()
+
+	if err != nil {
+		return err
+	}
+
+	if p.eof {
+		// this happens if the Caddyfile consists of only
+		// a line of addresses and nothing else
+		return nil
+	}
+
+	if ok, name := p.isSnippet(); ok {
+		if p.definedSnippets == nil {
+			p.definedSnippets = map[string][]Token{}
+		}
+		if _, found := p.definedSnippets[name]; found {
+			return p.Errf("redeclaration of previously declared snippet %s", name)
+		}
+		// consume all tokens til matched close brace
+		tokens, err := p.snippetTokens()
+		if err != nil {
+			return err
+		}
+		p.definedSnippets[name] = tokens
+		// empty block keys so we don't save this block as a real server.
+		p.block.Keys = nil
+		return nil
+	}
+
+	return p.blockContents()
+}
+
+func (p *parser) addresses() error {
+	var expectingAnother bool
+
+	for {
+		tkn := replaceEnvVars(p.Val())
+
+		// special case: import directive replaces tokens during parse-time
+		if tkn == "import" && p.isNewLine() {
+			err := p.doImport()
+			if err != nil {
+				return err
+			}
+			continue
+		}
+
+		// Open brace definitely indicates end of addresses
+		if tkn == "{" {
+			if expectingAnother {
+				return p.Errf("Expected another address but had '%s' - check for extra comma", tkn)
+			}
+			break
+		}
+
+		if tkn != "" { // empty token possible if user typed ""
+			// Trailing comma indicates another address will follow, which
+			// may possibly be on the next line
+			if tkn[len(tkn)-1] == ',' {
+				tkn = tkn[:len(tkn)-1]
+				expectingAnother = true
+			} else {
+				expectingAnother = false // but we may still see another one on this line
+			}
+
+			p.block.Keys = append(p.block.Keys, tkn)
+		}
+
+		// Advance token and possibly break out of loop or return error
+		hasNext := p.Next()
+		if expectingAnother && !hasNext {
+			return p.EOFErr()
+		}
+		if !hasNext {
+			p.eof = true
+			break // EOF
+		}
+		if !expectingAnother && p.isNewLine() {
+			break
+		}
+	}
+
+	return nil
+}
+
+func (p *parser) blockContents() error {
+	errOpenCurlyBrace := p.openCurlyBrace()
+	if errOpenCurlyBrace != nil {
+		// single-server configs don't need curly braces
+		p.cursor--
+	}
+
+	err := p.directives()
+	if err != nil {
+		return err
+	}
+
+	// only look for close curly brace if there was an opening
+	if errOpenCurlyBrace == nil {
+		err = p.closeCurlyBrace()
+		if err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+// directives parses through all the lines for directives
+// and it expects the next token to be the first
+// directive. It goes until EOF or closing curly brace
+// which ends the server block.
+func (p *parser) directives() error {
+	for p.Next() {
+		// end of server block
+		if p.Val() == "}" {
+			// p.nesting has already been decremented
+			break
+		}
+
+		// special case: import directive replaces tokens during parse-time
+		if p.Val() == "import" {
+			err := p.doImport()
+			if err != nil {
+				return err
+			}
+			p.cursor-- // cursor is advanced when we continue, so roll back one more
+			continue
+		}
+
+		// normal case: parse a directive as a new segment
+		// (a "segment" is a line which starts with a directive
+		// and which ends at the end of the line or at the end of
+		// the block that is opened at the end of the line)
+		if err := p.directive(); err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
+// doImport swaps out the import directive and its argument
+// (a total of 2 tokens) with the tokens in the specified file
+// or globbing pattern. When the function returns, the cursor
+// is on the token before where the import directive was. In
+// other words, call Next() to access the first token that was
+// imported.
+func (p *parser) doImport() error {
+	// syntax checks
+	if !p.NextArg() {
+		return p.ArgErr()
+	}
+	importPattern := replaceEnvVars(p.Val())
+	if importPattern == "" {
+		return p.Err("Import requires a non-empty filepath")
+	}
+	if p.NextArg() {
+		return p.Err("Import takes only one argument (glob pattern or file)")
+	}
+	// splice out the import directive and its argument (2 tokens total)
+	tokensBefore := p.tokens[:p.cursor-1]
+	tokensAfter := p.tokens[p.cursor+1:]
+	var importedTokens []Token
+
+	// first check snippets. That is a simple, non-recursive replacement
+	if p.definedSnippets != nil && p.definedSnippets[importPattern] != nil {
+		importedTokens = p.definedSnippets[importPattern]
+	} else {
+		// make path relative to the file of the _token_ being processed rather
+		// than current working directory (issue #867) and then use glob to get
+		// list of matching filenames
+		absFile, err := filepath.Abs(p.Dispenser.File())
+		if err != nil {
+			return p.Errf("Failed to get absolute path of file: %s: %v", p.Dispenser.File(), err)
+		}
+
+		var matches []string
+		var globPattern string
+		if !filepath.IsAbs(importPattern) {
+			globPattern = filepath.Join(filepath.Dir(absFile), importPattern)
+		} else {
+			globPattern = importPattern
+		}
+		if strings.Count(globPattern, "*") > 1 || strings.Count(globPattern, "?") > 1 ||
+			(strings.Contains(globPattern, "[") && strings.Contains(globPattern, "]")) {
+			// See issue #2096 - a pattern with many glob expansions can hang for too long
+			return p.Errf("Glob pattern may only contain one wildcard (*), but has others: %s", globPattern)
+		}
+		matches, err = filepath.Glob(globPattern)
+
+		if err != nil {
+			return p.Errf("Failed to use import pattern %s: %v", importPattern, err)
+		}
+		if len(matches) == 0 {
+			if strings.ContainsAny(globPattern, "*?[]") {
+				log.Printf("[WARNING] No files matching import glob pattern: %s", importPattern)
+			} else {
+				return p.Errf("File to import not found: %s", importPattern)
+			}
+		}
+
+		// collect all the imported tokens
+
+		for _, importFile := range matches {
+			newTokens, err := p.doSingleImport(importFile)
+			if err != nil {
+				return err
+			}
+			importedTokens = append(importedTokens, newTokens...)
+		}
+	}
+
+	// splice the imported tokens in the place of the import statement
+	// and rewind cursor so Next() will land on first imported token
+	p.tokens = append(tokensBefore, append(importedTokens, tokensAfter...)...)
+	p.cursor--
+
+	return nil
+}
+
+// doSingleImport lexes the individual file at importFile and returns
+// its tokens or an error, if any.
+func (p *parser) doSingleImport(importFile string) ([]Token, error) {
+	file, err := os.Open(importFile)
+	if err != nil {
+		return nil, p.Errf("Could not import %s: %v", importFile, err)
+	}
+	defer file.Close()
+
+	if info, err := file.Stat(); err != nil {
+		return nil, p.Errf("Could not import %s: %v", importFile, err)
+	} else if info.IsDir() {
+		return nil, p.Errf("Could not import %s: is a directory", importFile)
+	}
+
+	importedTokens, err := allTokens(importFile, file)
+	if err != nil {
+		return nil, p.Errf("Could not read tokens while importing %s: %v", importFile, err)
+	}
+
+	// Tack the file path onto these tokens so errors show the imported file's name
+	// (we use full, absolute path to avoid bugs: issue #1892)
+	filename, err := filepath.Abs(importFile)
+	if err != nil {
+		return nil, p.Errf("Failed to get absolute path of file: %s: %v", importFile, err)
+	}
+	for i := 0; i < len(importedTokens); i++ {
+		importedTokens[i].File = filename
+	}
+
+	return importedTokens, nil
+}
+
+// directive collects tokens until the directive's scope
+// closes (either end of line or end of curly brace block).
+// It expects the currently-loaded token to be a directive
+// (or } that ends a server block). The collected tokens
+// are loaded into the current server block for later use
+// by directive setup functions.
+func (p *parser) directive() error {
+	// evaluate any env vars in directive token
+	p.tokens[p.cursor].Text = replaceEnvVars(p.tokens[p.cursor].Text)
+
+	// a segment is a list of tokens associated with this directive
+	var segment Segment
+
+	// the directive itself is appended as a relevant token
+	segment = append(segment, p.Token())
+
+	for p.Next() {
+		if p.Val() == "{" {
+			p.nesting++
+		} else if p.isNewLine() && p.nesting == 0 {
+			p.cursor-- // read too far
+			break
+		} else if p.Val() == "}" && p.nesting > 0 {
+			p.nesting--
+		} else if p.Val() == "}" && p.nesting == 0 {
+			return p.Err("Unexpected '}' because no matching opening brace")
+		} else if p.Val() == "import" && p.isNewLine() {
+			if err := p.doImport(); err != nil {
+				return err
+			}
+			p.cursor-- // cursor is advanced when we continue, so roll back one more
+			continue
+		}
+		p.tokens[p.cursor].Text = replaceEnvVars(p.tokens[p.cursor].Text)
+		segment = append(segment, p.Token())
+	}
+
+	p.block.Segments = append(p.block.Segments, segment)
+
+	if p.nesting > 0 {
+		return p.EOFErr()
+	}
+
+	return nil
+}
+
+// openCurlyBrace expects the current token to be an
+// opening curly brace. This acts like an assertion
+// because it returns an error if the token is not
+// a opening curly brace. It does NOT advance the token.
+func (p *parser) openCurlyBrace() error {
+	if p.Val() != "{" {
+		return p.SyntaxErr("{")
+	}
+	return nil
+}
+
+// closeCurlyBrace expects the current token to be
+// a closing curly brace. This acts like an assertion
+// because it returns an error if the token is not
+// a closing curly brace. It does NOT advance the token.
+func (p *parser) closeCurlyBrace() error {
+	if p.Val() != "}" {
+		return p.SyntaxErr("}")
+	}
+	return nil
+}
+
+// replaceEnvVars replaces environment variables that appear in the token
+// and understands both the $UNIX and %WINDOWS% syntaxes.
+func replaceEnvVars(s string) string {
+	s = replaceEnvReferences(s, "{%", "%}")
+	s = replaceEnvReferences(s, "{$", "}")
+	return s
+}
+
+// replaceEnvReferences performs the actual replacement of env variables
+// in s, given the placeholder start and placeholder end strings.
+func replaceEnvReferences(s, refStart, refEnd string) string {
+	index := strings.Index(s, refStart)
+	for index != -1 {
+		endIndex := strings.Index(s[index:], refEnd)
+		if endIndex == -1 {
+			break
+		}
+
+		endIndex += index
+		if endIndex > index+len(refStart) {
+			ref := s[index : endIndex+len(refEnd)]
+			s = strings.Replace(s, ref, os.Getenv(ref[len(refStart):len(ref)-len(refEnd)]), -1)
+		} else {
+			return s
+		}
+		index = strings.Index(s, refStart)
+	}
+	return s
+}
+
+func (p *parser) isSnippet() (bool, string) {
+	keys := p.block.Keys
+	// A snippet block is a single key with parens. Nothing else qualifies.
+	if len(keys) == 1 && strings.HasPrefix(keys[0], "(") && strings.HasSuffix(keys[0], ")") {
+		return true, strings.TrimSuffix(keys[0][1:], ")")
+	}
+	return false, ""
+}
+
+// read and store everything in a block for later replay.
+func (p *parser) snippetTokens() ([]Token, error) {
+	// snippet must have curlies.
+	err := p.openCurlyBrace()
+	if err != nil {
+		return nil, err
+	}
+	nesting := 1 // count our own nesting in snippets
+	tokens := []Token{}
+	for p.Next() {
+		if p.Val() == "}" {
+			nesting--
+			if nesting == 0 {
+				break
+			}
+		}
+		if p.Val() == "{" {
+			nesting++
+		}
+		tokens = append(tokens, p.tokens[p.cursor])
+	}
+	// make sure we're matched up
+	if nesting != 0 {
+		return nil, p.SyntaxErr("}")
+	}
+	return tokens, nil
+}
+
+// ServerBlock associates any number of keys from the
+// head of the server block with tokens, which are
+// grouped by segments.
+type ServerBlock struct {
+	Keys     []string
+	Segments []Segment
+}
+
+// DispenseDirective returns a dispenser that contains
+// all the tokens in the server block.
+func (sb ServerBlock) DispenseDirective(dir string) *Dispenser {
+	var tokens []Token
+	for _, seg := range sb.Segments {
+		if len(seg) > 0 && seg[0].Text == dir {
+			tokens = append(tokens, seg...)
+		}
+	}
+	return NewDispenser(tokens)
+}
+
+// Segment is a list of tokens which begins with a directive
+// and ends at the end of the directive (either at the end of
+// the line, or at the end of a block it opens).
+type Segment []Token
+
+// Directive returns the directive name for the segment.
+// The directive name is the text of the first token.
+func (s Segment) Directive() string {
+	if len(s) > 0 {
+		return s[0].Text
+	}
+	return ""
+}

caddyconfig/caddyfile/parse_test.go

@@ -0,0 +1,679 @@
+// Copyright 2015 Light Code Labs, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddyfile
+
+import (
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+func TestAllTokens(t *testing.T) {
+	input := strings.NewReader("a b c\nd e")
+	expected := []string{"a", "b", "c", "d", "e"}
+	tokens, err := allTokens("TestAllTokens", input)
+
+	if err != nil {
+		t.Fatalf("Expected no error, got %v", err)
+	}
+	if len(tokens) != len(expected) {
+		t.Fatalf("Expected %d tokens, got %d", len(expected), len(tokens))
+	}
+
+	for i, val := range expected {
+		if tokens[i].Text != val {
+			t.Errorf("Token %d should be '%s' but was '%s'", i, val, tokens[i].Text)
+		}
+	}
+}
+
+func TestParseOneAndImport(t *testing.T) {
+	testParseOne := func(input string) (ServerBlock, error) {
+		p := testParser(input)
+		p.Next() // parseOne doesn't call Next() to start, so we must
+		err := p.parseOne()
+		return p.block, err
+	}
+
+	for i, test := range []struct {
+		input     string
+		shouldErr bool
+		keys      []string
+		numTokens []int // number of tokens to expect in each segment
+	}{
+		{`localhost`, false, []string{
+			"localhost",
+		}, []int{}},
+
+		{`localhost
+		  dir1`, false, []string{
+			"localhost",
+		}, []int{1}},
+
+		{`localhost:1234
+		  dir1 foo bar`, false, []string{
+			"localhost:1234",
+		}, []int{3},
+		},
+
+		{`localhost {
+		    dir1
+		  }`, false, []string{
+			"localhost",
+		}, []int{1}},
+
+		{`localhost:1234 {
+		    dir1 foo bar
+		    dir2
+		  }`, false, []string{
+			"localhost:1234",
+		}, []int{3, 1}},
+
+		{`http://localhost https://localhost
+		  dir1 foo bar`, false, []string{
+			"http://localhost",
+			"https://localhost",
+		}, []int{3}},
+
+		{`http://localhost https://localhost {
+		    dir1 foo bar
+		  }`, false, []string{
+			"http://localhost",
+			"https://localhost",
+		}, []int{3}},
+
+		{`http://localhost, https://localhost {
+		    dir1 foo bar
+		  }`, false, []string{
+			"http://localhost",
+			"https://localhost",
+		}, []int{3}},
+
+		{`http://localhost, {
+		  }`, true, []string{
+			"http://localhost",
+		}, []int{}},
+
+		{`host1:80, http://host2.com
+		  dir1 foo bar
+		  dir2 baz`, false, []string{
+			"host1:80",
+			"http://host2.com",
+		}, []int{3, 2}},
+
+		{`http://host1.com,
+		  http://host2.com,
+		  https://host3.com`, false, []string{
+			"http://host1.com",
+			"http://host2.com",
+			"https://host3.com",
+		}, []int{}},
+
+		{`http://host1.com:1234, https://host2.com
+		  dir1 foo {
+		    bar baz
+		  }
+		  dir2`, false, []string{
+			"http://host1.com:1234",
+			"https://host2.com",
+		}, []int{6, 1}},
+
+		{`127.0.0.1
+		  dir1 {
+		    bar baz
+		  }
+		  dir2 {
+		    foo bar
+		  }`, false, []string{
+			"127.0.0.1",
+		}, []int{5, 5}},
+
+		{`localhost
+		  dir1 {
+		    foo`, true, []string{
+			"localhost",
+		}, []int{3}},
+
+		{`localhost
+		  dir1 {
+		  }`, false, []string{
+			"localhost",
+		}, []int{3}},
+
+		{`localhost
+		  dir1 {
+		  } }`, true, []string{
+			"localhost",
+		}, []int{}},
+
+		{`localhost
+		  dir1 {
+		    nested {
+		      foo
+		    }
+		  }
+		  dir2 foo bar`, false, []string{
+			"localhost",
+		}, []int{7, 3}},
+
+		{``, false, []string{}, []int{}},
+
+		{`localhost
+		  dir1 arg1
+		  import testdata/import_test1.txt`, false, []string{
+			"localhost",
+		}, []int{2, 3, 1}},
+
+		{`import testdata/import_test2.txt`, false, []string{
+			"host1",
+		}, []int{1, 2}},
+
+		{`import testdata/import_test1.txt testdata/import_test2.txt`, true, []string{}, []int{}},
+
+		{`import testdata/not_found.txt`, true, []string{}, []int{}},
+
+		{`""`, false, []string{}, []int{}},
+
+		{``, false, []string{}, []int{}},
+
+		// test cases found by fuzzing!
+		{`import }{$"`, true, []string{}, []int{}},
+		{`import /*/*.txt`, true, []string{}, []int{}},
+		{`import /???/?*?o`, true, []string{}, []int{}},
+		{`import /??`, true, []string{}, []int{}},
+		{`import /[a-z]`, true, []string{}, []int{}},
+		{`import {$}`, true, []string{}, []int{}},
+		{`import {%}`, true, []string{}, []int{}},
+		{`import {$$}`, true, []string{}, []int{}},
+		{`import {%%}`, true, []string{}, []int{}},
+	} {
+		result, err := testParseOne(test.input)
+
+		if test.shouldErr && err == nil {
+			t.Errorf("Test %d: Expected an error, but didn't get one", i)
+		}
+		if !test.shouldErr && err != nil {
+			t.Errorf("Test %d: Expected no error, but got: %v", i, err)
+		}
+
+		if len(result.Keys) != len(test.keys) {
+			t.Errorf("Test %d: Expected %d keys, got %d",
+				i, len(test.keys), len(result.Keys))
+			continue
+		}
+		for j, addr := range result.Keys {
+			if addr != test.keys[j] {
+				t.Errorf("Test %d, key %d: Expected '%s', but was '%s'",
+					i, j, test.keys[j], addr)
+			}
+		}
+
+		if len(result.Segments) != len(test.numTokens) {
+			t.Errorf("Test %d: Expected %d segments, had %d",
+				i, len(test.numTokens), len(result.Segments))
+			continue
+		}
+
+		for j, seg := range result.Segments {
+			if len(seg) != test.numTokens[j] {
+				t.Errorf("Test %d, segment %d: Expected %d tokens, counted %d",
+					i, j, test.numTokens[j], len(seg))
+				continue
+			}
+		}
+	}
+}
+
+func TestRecursiveImport(t *testing.T) {
+	testParseOne := func(input string) (ServerBlock, error) {
+		p := testParser(input)
+		p.Next() // parseOne doesn't call Next() to start, so we must
+		err := p.parseOne()
+		return p.block, err
+	}
+
+	isExpected := func(got ServerBlock) bool {
+		if len(got.Keys) != 1 || got.Keys[0] != "localhost" {
+			t.Errorf("got keys unexpected: expect localhost, got %v", got.Keys)
+			return false
+		}
+		if len(got.Segments) != 2 {
+			t.Errorf("got wrong number of segments: expect 2, got %d", len(got.Segments))
+			return false
+		}
+		if len(got.Segments[0]) != 1 || len(got.Segments[1]) != 2 {
+			t.Errorf("got unexpect tokens: %v", got.Segments)
+			return false
+		}
+		return true
+	}
+
+	recursiveFile1, err := filepath.Abs("testdata/recursive_import_test1")
+	if err != nil {
+		t.Fatal(err)
+	}
+	recursiveFile2, err := filepath.Abs("testdata/recursive_import_test2")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// test relative recursive import
+	err = ioutil.WriteFile(recursiveFile1, []byte(
+		`localhost
+		dir1
+		import recursive_import_test2`), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.Remove(recursiveFile1)
+
+	err = ioutil.WriteFile(recursiveFile2, []byte("dir2 1"), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.Remove(recursiveFile2)
+
+	// import absolute path
+	result, err := testParseOne("import " + recursiveFile1)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !isExpected(result) {
+		t.Error("absolute+relative import failed")
+	}
+
+	// import relative path
+	result, err = testParseOne("import testdata/recursive_import_test1")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !isExpected(result) {
+		t.Error("relative+relative import failed")
+	}
+
+	// test absolute recursive import
+	err = ioutil.WriteFile(recursiveFile1, []byte(
+		`localhost
+		dir1
+		import `+recursiveFile2), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	// import absolute path
+	result, err = testParseOne("import " + recursiveFile1)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !isExpected(result) {
+		t.Error("absolute+absolute import failed")
+	}
+
+	// import relative path
+	result, err = testParseOne("import testdata/recursive_import_test1")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !isExpected(result) {
+		t.Error("relative+absolute import failed")
+	}
+}
+
+func TestDirectiveImport(t *testing.T) {
+	testParseOne := func(input string) (ServerBlock, error) {
+		p := testParser(input)
+		p.Next() // parseOne doesn't call Next() to start, so we must
+		err := p.parseOne()
+		return p.block, err
+	}
+
+	isExpected := func(got ServerBlock) bool {
+		if len(got.Keys) != 1 || got.Keys[0] != "localhost" {
+			t.Errorf("got keys unexpected: expect localhost, got %v", got.Keys)
+			return false
+		}
+		if len(got.Segments) != 2 {
+			t.Errorf("got wrong number of segments: expect 2, got %d", len(got.Segments))
+			return false
+		}
+		if len(got.Segments[0]) != 1 || len(got.Segments[1]) != 8 {
+			t.Errorf("got unexpect tokens: %v", got.Segments)
+			return false
+		}
+		return true
+	}
+
+	directiveFile, err := filepath.Abs("testdata/directive_import_test")
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	err = ioutil.WriteFile(directiveFile, []byte(`prop1 1
+	prop2 2`), 0644)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer os.Remove(directiveFile)
+
+	// import from existing file
+	result, err := testParseOne(`localhost
+	dir1
+	proxy {
+		import testdata/directive_import_test
+		transparent
+	}`)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !isExpected(result) {
+		t.Error("directive import failed")
+	}
+
+	// import from nonexistent file
+	_, err = testParseOne(`localhost
+	dir1
+	proxy {
+		import testdata/nonexistent_file
+		transparent
+	}`)
+	if err == nil {
+		t.Fatal("expected error when importing a nonexistent file")
+	}
+}
+
+func TestParseAll(t *testing.T) {
+	for i, test := range []struct {
+		input     string
+		shouldErr bool
+		keys      [][]string // keys per server block, in order
+	}{
+		{`localhost`, false, [][]string{
+			{"localhost"},
+		}},
+
+		{`localhost:1234`, false, [][]string{
+			{"localhost:1234"},
+		}},
+
+		{`localhost:1234 {
+		  }
+		  localhost:2015 {
+		  }`, false, [][]string{
+			{"localhost:1234"},
+			{"localhost:2015"},
+		}},
+
+		{`localhost:1234, http://host2`, false, [][]string{
+			{"localhost:1234", "http://host2"},
+		}},
+
+		{`localhost:1234, http://host2,`, true, [][]string{}},
+
+		{`http://host1.com, http://host2.com {
+		  }
+		  https://host3.com, https://host4.com {
+		  }`, false, [][]string{
+			{"http://host1.com", "http://host2.com"},
+			{"https://host3.com", "https://host4.com"},
+		}},
+
+		{`import testdata/import_glob*.txt`, false, [][]string{
+			{"glob0.host0"},
+			{"glob0.host1"},
+			{"glob1.host0"},
+			{"glob2.host0"},
+		}},
+
+		{`import notfound/*`, false, [][]string{}},        // glob needn't error with no matches
+		{`import notfound/file.conf`, true, [][]string{}}, // but a specific file should
+	} {
+		p := testParser(test.input)
+		blocks, err := p.parseAll()
+
+		if test.shouldErr && err == nil {
+			t.Errorf("Test %d: Expected an error, but didn't get one", i)
+		}
+		if !test.shouldErr && err != nil {
+			t.Errorf("Test %d: Expected no error, but got: %v", i, err)
+		}
+
+		if len(blocks) != len(test.keys) {
+			t.Errorf("Test %d: Expected %d server blocks, got %d",
+				i, len(test.keys), len(blocks))
+			continue
+		}
+		for j, block := range blocks {
+			if len(block.Keys) != len(test.keys[j]) {
+				t.Errorf("Test %d: Expected %d keys in block %d, got %d",
+					i, len(test.keys[j]), j, len(block.Keys))
+				continue
+			}
+			for k, addr := range block.Keys {
+				if addr != test.keys[j][k] {
+					t.Errorf("Test %d, block %d, key %d: Expected '%s', but got '%s'",
+						i, j, k, test.keys[j][k], addr)
+				}
+			}
+		}
+	}
+}
+
+func TestEnvironmentReplacement(t *testing.T) {
+	os.Setenv("PORT", "8080")
+	os.Setenv("ADDRESS", "servername.com")
+	os.Setenv("FOOBAR", "foobar")
+	os.Setenv("PARTIAL_DIR", "r1")
+
+	// basic test; unix-style env vars
+	p := testParser(`{$ADDRESS}`)
+	blocks, _ := p.parseAll()
+	if actual, expected := blocks[0].Keys[0], "servername.com"; expected != actual {
+		t.Errorf("Expected key to be '%s' but was '%s'", expected, actual)
+	}
+
+	// basic test; unix-style env vars
+	p = testParser(`di{$PARTIAL_DIR}`)
+	blocks, _ = p.parseAll()
+	if actual, expected := blocks[0].Keys[0], "dir1"; expected != actual {
+		t.Errorf("Expected key to be '%s' but was '%s'", expected, actual)
+	}
+
+	// multiple vars per token
+	p = testParser(`{$ADDRESS}:{$PORT}`)
+	blocks, _ = p.parseAll()
+	if actual, expected := blocks[0].Keys[0], "servername.com:8080"; expected != actual {
+		t.Errorf("Expected key to be '%s' but was '%s'", expected, actual)
+	}
+
+	// windows-style var and unix style in same token
+	p = testParser(`{%ADDRESS%}:{$PORT}`)
+	blocks, _ = p.parseAll()
+	if actual, expected := blocks[0].Keys[0], "servername.com:8080"; expected != actual {
+		t.Errorf("Expected key to be '%s' but was '%s'", expected, actual)
+	}
+
+	// reverse order
+	p = testParser(`{$ADDRESS}:{%PORT%}`)
+	blocks, _ = p.parseAll()
+	if actual, expected := blocks[0].Keys[0], "servername.com:8080"; expected != actual {
+		t.Errorf("Expected key to be '%s' but was '%s'", expected, actual)
+	}
+
+	// env var in server block body as argument
+	p = testParser(":{%PORT%}\ndir1 {$FOOBAR}")
+	blocks, _ = p.parseAll()
+	if actual, expected := blocks[0].Keys[0], ":8080"; expected != actual {
+		t.Errorf("Expected key to be '%s' but was '%s'", expected, actual)
+	}
+	if actual, expected := blocks[0].Segments[0][1].Text, "foobar"; expected != actual {
+		t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
+	}
+
+	// combined windows env vars in argument
+	p = testParser(":{%PORT%}\ndir1 {%ADDRESS%}/{%FOOBAR%}")
+	blocks, _ = p.parseAll()
+	if actual, expected := blocks[0].Segments[0][1].Text, "servername.com/foobar"; expected != actual {
+		t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
+	}
+
+	// malformed env var (windows)
+	p = testParser(":1234\ndir1 {%ADDRESS}")
+	blocks, _ = p.parseAll()
+	if actual, expected := blocks[0].Segments[0][1].Text, "{%ADDRESS}"; expected != actual {
+		t.Errorf("Expected host to be '%s' but was '%s'", expected, actual)
+	}
+
+	// malformed (non-existent) env var (unix)
+	p = testParser(`:{$PORT$}`)
+	blocks, _ = p.parseAll()
+	if actual, expected := blocks[0].Keys[0], ":"; expected != actual {
+		t.Errorf("Expected key to be '%s' but was '%s'", expected, actual)
+	}
+
+	// in quoted field
+	p = testParser(":1234\ndir1 \"Test {$FOOBAR} test\"")
+	blocks, _ = p.parseAll()
+	if actual, expected := blocks[0].Segments[0][1].Text, "Test foobar test"; expected != actual {
+		t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
+	}
+
+	// after end token
+	p = testParser(":1234\nanswer \"{{ .Name }} {$FOOBAR}\"")
+	blocks, _ = p.parseAll()
+	if actual, expected := blocks[0].Segments[0][1].Text, "{{ .Name }} foobar"; expected != actual {
+		t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
+	}
+}
+
+func TestSnippets(t *testing.T) {
+	p := testParser(`
+		(common) {
+			gzip foo
+			errors stderr
+		}
+		http://example.com {
+			import common
+		}
+	`)
+	blocks, err := p.parseAll()
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, b := range blocks {
+		t.Log(b.Keys)
+		t.Log(b.Segments)
+	}
+	if len(blocks) != 1 {
+		t.Fatalf("Expect exactly one server block. Got %d.", len(blocks))
+	}
+	if actual, expected := blocks[0].Keys[0], "http://example.com"; expected != actual {
+		t.Errorf("Expected server name to be '%s' but was '%s'", expected, actual)
+	}
+	if len(blocks[0].Segments) != 2 {
+		t.Fatalf("Server block should have tokens from import, got: %+v", blocks[0])
+	}
+	if actual, expected := blocks[0].Segments[0][0].Text, "gzip"; expected != actual {
+		t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
+	}
+	if actual, expected := blocks[0].Segments[1][1].Text, "stderr"; expected != actual {
+		t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
+	}
+}
+
+func writeStringToTempFileOrDie(t *testing.T, str string) (pathToFile string) {
+	file, err := ioutil.TempFile("", t.Name())
+	if err != nil {
+		panic(err) // get a stack trace so we know where this was called from.
+	}
+	if _, err := file.WriteString(str); err != nil {
+		panic(err)
+	}
+	if err := file.Close(); err != nil {
+		panic(err)
+	}
+	return file.Name()
+}
+
+func TestImportedFilesIgnoreNonDirectiveImportTokens(t *testing.T) {
+	fileName := writeStringToTempFileOrDie(t, `
+		http://example.com {
+			# This isn't an import directive, it's just an arg with value 'import'
+			basicauth / import password
+		}
+	`)
+	// Parse the root file that imports the other one.
+	p := testParser(`import ` + fileName)
+	blocks, err := p.parseAll()
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, b := range blocks {
+		t.Log(b.Keys)
+		t.Log(b.Segments)
+	}
+	auth := blocks[0].Segments[0]
+	line := auth[0].Text + " " + auth[1].Text + " " + auth[2].Text + " " + auth[3].Text
+	if line != "basicauth / import password" {
+		// Previously, it would be changed to:
+		//   basicauth / import /path/to/test/dir/password
+		// referencing a file that (probably) doesn't exist and changing the
+		// password!
+		t.Errorf("Expected basicauth tokens to be 'basicauth / import password' but got %#q", line)
+	}
+}
+
+func TestSnippetAcrossMultipleFiles(t *testing.T) {
+	// Make the derived Caddyfile that expects (common) to be defined.
+	fileName := writeStringToTempFileOrDie(t, `
+		http://example.com {
+			import common
+		}
+	`)
+
+	// Parse the root file that defines (common) and then imports the other one.
+	p := testParser(`
+		(common) {
+			gzip foo
+		}
+		import ` + fileName + `
+	`)
+
+	blocks, err := p.parseAll()
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, b := range blocks {
+		t.Log(b.Keys)
+		t.Log(b.Segments)
+	}
+	if len(blocks) != 1 {
+		t.Fatalf("Expect exactly one server block. Got %d.", len(blocks))
+	}
+	if actual, expected := blocks[0].Keys[0], "http://example.com"; expected != actual {
+		t.Errorf("Expected server name to be '%s' but was '%s'", expected, actual)
+	}
+	if len(blocks[0].Segments) != 1 {
+		t.Fatalf("Server block should have tokens from import")
+	}
+	if actual, expected := blocks[0].Segments[0][0].Text, "gzip"; expected != actual {
+		t.Errorf("Expected argument to be '%s' but was '%s'", expected, actual)
+	}
+}
+
+func testParser(input string) parser {
+	return parser{Dispenser: newTestDispenser(input)}
+}

caddyconfig/caddyfile/testdata/import_glob0.txt

@@ -0,0 +1,6 @@
+glob0.host0 {
+	dir2 arg1
+}
+
+glob0.host1 {
+}

caddyconfig/caddyfile/testdata/import_glob1.txt

@@ -0,0 +1,4 @@
+glob1.host0 {
+	dir1
+	dir2 arg1
+}

caddyconfig/caddyfile/testdata/import_glob2.txt

@@ -0,0 +1,3 @@
+glob2.host0 {
+	dir2 arg1
+}

caddyconfig/caddyfile/testdata/import_test1.txt

@@ -0,0 +1,2 @@
+dir2 arg1 arg2
+dir3

caddyconfig/caddyfile/testdata/import_test2.txt

@@ -0,0 +1,4 @@
+host1 {
+	dir1
+	dir2 arg1
+}

caddyconfig/configadapters.go

@@ -0,0 +1,117 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddyconfig
+
+import (
+	"encoding/json"
+	"fmt"
+)
+
+// Adapter is a type which can adapt a configuration to Caddy JSON.
+// It returns the results and any warnings, or an error.
+type Adapter interface {
+	Adapt(body []byte, options map[string]interface{}) ([]byte, []Warning, error)
+}
+
+// Warning represents a warning or notice related to conversion.
+type Warning struct {
+	File      string `json:"file,omitempty"`
+	Line      int    `json:"line,omitempty"`
+	Directive string `json:"directive,omitempty"`
+	Message   string `json:"message,omitempty"`
+}
+
+// JSON encodes val as JSON, returning it as a json.RawMessage. Any
+// marshaling errors (which are highly unlikely with correct code)
+// are converted to warnings. This is convenient when filling config
+// structs that require a json.RawMessage, without having to worry
+// about errors.
+func JSON(val interface{}, warnings *[]Warning) json.RawMessage {
+	b, err := json.Marshal(val)
+	if err != nil {
+		if warnings != nil {
+			*warnings = append(*warnings, Warning{Message: err.Error()})
+		}
+		return nil
+	}
+	return b
+}
+
+// JSONModuleObject is like JSON, except it marshals val into a JSON object
+// and then adds a key to that object named fieldName with the value fieldVal.
+// This is useful for JSON-encoding module values where the module name has to
+// be described within the object by a certain key; for example,
+// "responder": "file_server" for a file server HTTP responder. The val must
+// encode into a map[string]interface{} (i.e. it must be a struct or map),
+// and any errors are converted into warnings, so this can be conveniently
+// used when filling a struct. For correct code, there should be no errors.
+func JSONModuleObject(val interface{}, fieldName, fieldVal string, warnings *[]Warning) json.RawMessage {
+	// encode to a JSON object first
+	enc, err := json.Marshal(val)
+	if err != nil {
+		if warnings != nil {
+			*warnings = append(*warnings, Warning{Message: err.Error()})
+		}
+		return nil
+	}
+
+	// then decode the object
+	var tmp map[string]interface{}
+	err = json.Unmarshal(enc, &tmp)
+	if err != nil {
+		if warnings != nil {
+			*warnings = append(*warnings, Warning{Message: err.Error()})
+		}
+		return nil
+	}
+
+	// so we can easily add the module's field with its appointed value
+	tmp[fieldName] = fieldVal
+
+	// then re-marshal as JSON
+	result, err := json.Marshal(tmp)
+	if err != nil {
+		if warnings != nil {
+			*warnings = append(*warnings, Warning{Message: err.Error()})
+		}
+		return nil
+	}
+
+	return result
+}
+
+// JSONIndent is used to JSON-marshal the final resulting Caddy
+// configuration in a consistent, human-readable way.
+func JSONIndent(val interface{}) ([]byte, error) {
+	return json.MarshalIndent(val, "", "\t")
+}
+
+// RegisterAdapter registers a config adapter with the given name.
+// This should usually be done at init-time.
+func RegisterAdapter(name string, adapter Adapter) error {
+	if _, ok := configAdapters[name]; ok {
+		return fmt.Errorf("%s: already registered", name)
+	}
+	configAdapters[name] = adapter
+	return nil
+}
+
+// GetAdapter returns the adapter with the given name,
+// or nil if one with that name is not registered.
+func GetAdapter(name string) Adapter {
+	return configAdapters[name]
+}
+
+var configAdapters = make(map[string]Adapter)

caddyconfig/httpcaddyfile/addresses.go

@@ -0,0 +1,334 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpcaddyfile
+
+import (
+	"fmt"
+	"net"
+	"net/url"
+	"reflect"
+	"strconv"
+	"strings"
+
+	"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
+	"github.com/caddyserver/caddy/v2/modules/caddyhttp"
+	"github.com/mholt/certmagic"
+)
+
+// mapAddressToServerBlocks returns a map of listener address to list of server
+// blocks that will be served on that address. To do this, each server block is
+// expanded so that each one is considered individually, although keys of a
+// server block that share the same address stay grouped together so the config
+// isn't repeated unnecessarily. For example, this Caddyfile:
+//
+// 	example.com {
+// 		bind 127.0.0.1
+// 	}
+// 	www.example.com, example.net/path, localhost:9999 {
+// 		bind 127.0.0.1 1.2.3.4
+// 	}
+//
+// has two server blocks to start with. But expressed in this Caddyfile are
+// actually 4 listener addresses: 127.0.0.1:443, 1.2.3.4:443, 127.0.0.1:9999,
+// and 127.0.0.1:9999. This is because the bind directive is applied to each
+// key of its server block (specifying the host part), and each key may have
+// a different port. And we definitely need to be sure that a site which is
+// bound to be served on a specific interface is not served on others just
+// beceause that is more convenient: it would be a potential security risk
+// if the difference between interfaces means private vs. public.
+//
+// So what this function does for the example above is iterate each server
+// block, and for each server block, iterate its keys. For the first, it
+// finds one key (example.com) and determines its listener address
+// (127.0.0.1:443 - because of 'bind' and automatic HTTPS). It then adds
+// the listener address to the map value returned by this function, with
+// the first server block as one of its associations.
+//
+// It then iterates each key on the second server block and associates them
+// with one or more listener addresses. Indeed, each key in this block has
+// two listener addresses because of the 'bind' directive. Once we know
+// which addresses serve which keys, we can create a new server block for
+// each address containing the contents of the server block and only those
+// specific keys of the server block which use that address.
+//
+// It is possible and even likely that some keys in the returned map have
+// the exact same list of server blocks (i.e. they are identical). This
+// happens when multiple hosts are declared with a 'bind' directive and
+// the resulting listener addresses are not shared by any other server
+// block (or the other server blocks are exactly identical in their token
+// contents). This happens with our example above because 1.2.3.4:443
+// and 1.2.3.4:9999 are used exclusively with the second server block. This
+// repetition may be undesirable, so call consolidateAddrMappings() to map
+// multiple addresses to the same lists of server blocks (a many:many mapping).
+// (Doing this is essentially a map-reduce technique.)
+func (st *ServerType) mapAddressToServerBlocks(originalServerBlocks []serverBlock) (map[string][]serverBlock, error) {
+	sbmap := make(map[string][]serverBlock)
+
+	for i, sblock := range originalServerBlocks {
+		// within a server block, we need to map all the listener addresses
+		// implied by the server block to the keys of the server block which
+		// will be served by them; this has the effect of treating each
+		// key of a server block as its own, but without having to repeat its
+		// contents in cases where multiple keys really can be served together
+		addrToKeys := make(map[string][]string)
+		for j, key := range sblock.block.Keys {
+			// a key can have multiple listener addresses if there are multiple
+			// arguments to the 'bind' directive (although they will all have
+			// the same port, since the port is defined by the key or is implicit
+			// through automatic HTTPS)
+			addrs, err := st.listenerAddrsForServerBlockKey(sblock, key)
+			if err != nil {
+				return nil, fmt.Errorf("server block %d, key %d (%s): determining listener address: %v", i, j, key, err)
+			}
+
+			// associate this key with each listener address it is served on
+			for _, addr := range addrs {
+				addrToKeys[addr] = append(addrToKeys[addr], key)
+			}
+		}
+
+		// now that we know which addresses serve which keys of this
+		// server block, we iterate that mapping and create a list of
+		// new server blocks for each address where the keys of the
+		// server block are only the ones which use the address; but
+		// the contents (tokens) are of course the same
+		for addr, keys := range addrToKeys {
+			sbmap[addr] = append(sbmap[addr], serverBlock{
+				block: caddyfile.ServerBlock{
+					Keys:     keys,
+					Segments: sblock.block.Segments,
+				},
+				pile: sblock.pile,
+			})
+		}
+	}
+
+	return sbmap, nil
+}
+
+// consolidateAddrMappings eliminates repetition of identical server blocks in a mapping of
+// single listener addresses to lists of server blocks. Since multiple addresses may serve
+// identical sites (server block contents), this function turns a 1:many mapping into a
+// many:many mapping. Server block contents (tokens) must be exactly identical so that
+// reflect.DeepEqual returns true in order for the addresses to be combined. Identical
+// entries are deleted from the addrToServerBlocks map. Essentially, each pairing (each
+// association from multiple addresses to multiple server blocks; i.e. each element of
+// the returned slice) becomes a server definition in the output JSON.
+func (st *ServerType) consolidateAddrMappings(addrToServerBlocks map[string][]serverBlock) []sbAddrAssociation {
+	var sbaddrs []sbAddrAssociation
+	for addr, sblocks := range addrToServerBlocks {
+		// we start with knowing that at least this address
+		// maps to these server blocks
+		a := sbAddrAssociation{
+			addresses:    []string{addr},
+			serverBlocks: sblocks,
+		}
+
+		// now find other addresses that map to identical
+		// server blocks and add them to our list of
+		// addresses, while removing them from the map
+		for otherAddr, otherSblocks := range addrToServerBlocks {
+			if addr == otherAddr {
+				continue
+			}
+			if reflect.DeepEqual(sblocks, otherSblocks) {
+				a.addresses = append(a.addresses, otherAddr)
+				delete(addrToServerBlocks, otherAddr)
+			}
+		}
+
+		sbaddrs = append(sbaddrs, a)
+	}
+	return sbaddrs
+}
+
+func (st *ServerType) listenerAddrsForServerBlockKey(sblock serverBlock, key string) ([]string, error) {
+	addr, err := ParseAddress(key)
+	if err != nil {
+		return nil, fmt.Errorf("parsing key: %v", err)
+	}
+	addr = addr.Normalize()
+
+	lnPort := DefaultPort
+	if addr.Port != "" {
+		// port explicitly defined
+		lnPort = addr.Port
+	} else if certmagic.HostQualifies(addr.Host) {
+		// automatic HTTPS
+		lnPort = strconv.Itoa(certmagic.HTTPSPort)
+	}
+
+	// the bind directive specifies hosts, but is optional
+	var lnHosts []string
+	for _, cfgVal := range sblock.pile["bind"] {
+		lnHosts = append(lnHosts, cfgVal.Value.([]string)...)
+	}
+	if len(lnHosts) == 0 {
+		lnHosts = []string{""}
+	}
+
+	// use a map to prevent duplication
+	listeners := make(map[string]struct{})
+	for _, host := range lnHosts {
+		listeners[net.JoinHostPort(host, lnPort)] = struct{}{}
+	}
+
+	// now turn map into list
+	var listenersList []string
+	for lnStr := range listeners {
+		listenersList = append(listenersList, lnStr)
+	}
+	// sort.Strings(listenersList) // TODO: is sorting necessary?
+
+	return listenersList, nil
+}
+
+// Address represents a site address. It contains
+// the original input value, and the component
+// parts of an address. The component parts may be
+// updated to the correct values as setup proceeds,
+// but the original value should never be changed.
+//
+// The Host field must be in a normalized form.
+type Address struct {
+	Original, Scheme, Host, Port, Path string
+}
+
+// ParseAddress parses an address string into a structured format with separate
+// scheme, host, port, and path portions, as well as the original input string.
+func ParseAddress(str string) (Address, error) {
+	httpPort, httpsPort := strconv.Itoa(certmagic.HTTPPort), strconv.Itoa(certmagic.HTTPSPort)
+
+	input := str
+
+	// Split input into components (prepend with // to force host portion by default)
+	if !strings.Contains(str, "//") && !strings.HasPrefix(str, "/") {
+		str = "//" + str
+	}
+
+	u, err := url.Parse(str)
+	if err != nil {
+		return Address{}, err
+	}
+
+	// separate host and port
+	host, port, err := net.SplitHostPort(u.Host)
+	if err != nil {
+		host, port, err = net.SplitHostPort(u.Host + ":")
+		if err != nil {
+			host = u.Host
+		}
+	}
+
+	// see if we can set port based off scheme
+	if port == "" {
+		if u.Scheme == "http" {
+			port = httpPort
+		} else if u.Scheme == "https" {
+			port = httpsPort
+		}
+	}
+
+	// error if scheme and port combination violate convention
+	if (u.Scheme == "http" && port == httpsPort) || (u.Scheme == "https" && port == httpPort) {
+		return Address{}, fmt.Errorf("[%s] scheme and port violate convention", input)
+	}
+
+	return Address{Original: input, Scheme: u.Scheme, Host: host, Port: port, Path: u.Path}, err
+}
+
+// TODO: which of the methods on Address are even used?
+
+// String returns a human-readable form of a. It will
+// be a cleaned-up and filled-out URL string.
+func (a Address) String() string {
+	if a.Host == "" && a.Port == "" {
+		return ""
+	}
+	scheme := a.Scheme
+	if scheme == "" {
+		if a.Port == strconv.Itoa(certmagic.HTTPSPort) {
+			scheme = "https"
+		} else {
+			scheme = "http"
+		}
+	}
+	s := scheme
+	if s != "" {
+		s += "://"
+	}
+	if a.Port != "" &&
+		((scheme == "https" && a.Port != strconv.Itoa(caddyhttp.DefaultHTTPSPort)) ||
+			(scheme == "http" && a.Port != strconv.Itoa(caddyhttp.DefaultHTTPPort))) {
+		s += net.JoinHostPort(a.Host, a.Port)
+	} else {
+		s += a.Host
+	}
+	if a.Path != "" {
+		s += a.Path
+	}
+	return s
+}
+
+// Normalize returns a normalized version of a.
+func (a Address) Normalize() Address {
+	path := a.Path
+	if !caseSensitivePath {
+		path = strings.ToLower(path)
+	}
+
+	// ensure host is normalized if it's an IP address
+	host := a.Host
+	if ip := net.ParseIP(host); ip != nil {
+		host = ip.String()
+	}
+
+	return Address{
+		Original: a.Original,
+		Scheme:   strings.ToLower(a.Scheme),
+		Host:     strings.ToLower(host),
+		Port:     a.Port,
+		Path:     path,
+	}
+}
+
+// Key returns a string form of a, much like String() does, but this
+// method doesn't add anything default that wasn't in the original.
+func (a Address) Key() string {
+	res := ""
+	if a.Scheme != "" {
+		res += a.Scheme + "://"
+	}
+	if a.Host != "" {
+		res += a.Host
+	}
+	// insert port only if the original has its own explicit port
+	if a.Port != "" &&
+		len(a.Original) >= len(res) &&
+		strings.HasPrefix(a.Original[len(res):], ":"+a.Port) {
+		res += ":" + a.Port
+	}
+	if a.Path != "" {
+		res += a.Path
+	}
+	return res
+}
+
+const (
+	// DefaultPort is the default port to use.
+	DefaultPort = "2015"
+
+	caseSensitivePath = false // TODO: Used?
+)

caddyconfig/httpcaddyfile/addresses_test.go

@@ -0,0 +1,166 @@
+package httpcaddyfile
+
+import (
+	"strings"
+	"testing"
+)
+
+func TestParseAddress(t *testing.T) {
+	for i, test := range []struct {
+		input                    string
+		scheme, host, port, path string
+		shouldErr                bool
+	}{
+		{`localhost`, "", "localhost", "", "", false},
+		{`localhost:1234`, "", "localhost", "1234", "", false},
+		{`localhost:`, "", "localhost", "", "", false},
+		{`0.0.0.0`, "", "0.0.0.0", "", "", false},
+		{`127.0.0.1:1234`, "", "127.0.0.1", "1234", "", false},
+		{`:1234`, "", "", "1234", "", false},
+		{`[::1]`, "", "::1", "", "", false},
+		{`[::1]:1234`, "", "::1", "1234", "", false},
+		{`:`, "", "", "", "", false},
+		{`:http`, "", "", "", "", true},
+		{`:https`, "", "", "", "", true},
+		{`localhost:http`, "", "", "", "", true}, // using service name in port is verboten, as of Go 1.12.8
+		{`localhost:https`, "", "", "", "", true},
+		{`http://localhost:https`, "", "", "", "", true}, // conflict
+		{`http://localhost:http`, "", "", "", "", true},  // repeated scheme
+		{`host:https/path`, "", "", "", "", true},
+		{`http://localhost:443`, "", "", "", "", true}, // not conventional
+		{`https://localhost:80`, "", "", "", "", true}, // not conventional
+		{`http://localhost`, "http", "localhost", "80", "", false},
+		{`https://localhost`, "https", "localhost", "443", "", false},
+		{`http://127.0.0.1`, "http", "127.0.0.1", "80", "", false},
+		{`https://127.0.0.1`, "https", "127.0.0.1", "443", "", false},
+		{`http://[::1]`, "http", "::1", "80", "", false},
+		{`http://localhost:1234`, "http", "localhost", "1234", "", false},
+		{`https://127.0.0.1:1234`, "https", "127.0.0.1", "1234", "", false},
+		{`http://[::1]:1234`, "http", "::1", "1234", "", false},
+		{``, "", "", "", "", false},
+		{`::1`, "", "::1", "", "", true},
+		{`localhost::`, "", "localhost::", "", "", true},
+		{`#$%@`, "", "", "", "", true},
+		{`host/path`, "", "host", "", "/path", false},
+		{`http://host/`, "http", "host", "80", "/", false},
+		{`//asdf`, "", "asdf", "", "", false},
+		{`:1234/asdf`, "", "", "1234", "/asdf", false},
+		{`http://host/path`, "http", "host", "80", "/path", false},
+		{`https://host:443/path/foo`, "https", "host", "443", "/path/foo", false},
+		{`host:80/path`, "", "host", "80", "/path", false},
+		{`/path`, "", "", "", "/path", false},
+	} {
+		actual, err := ParseAddress(test.input)
+
+		if err != nil && !test.shouldErr {
+			t.Errorf("Test %d (%s): Expected no error, but had error: %v", i, test.input, err)
+		}
+		if err == nil && test.shouldErr {
+			t.Errorf("Test %d (%s): Expected error, but had none", i, test.input)
+		}
+
+		if !test.shouldErr && actual.Original != test.input {
+			t.Errorf("Test %d (%s): Expected original '%s', got '%s'", i, test.input, test.input, actual.Original)
+		}
+		if actual.Scheme != test.scheme {
+			t.Errorf("Test %d (%s): Expected scheme '%s', got '%s'", i, test.input, test.scheme, actual.Scheme)
+		}
+		if actual.Host != test.host {
+			t.Errorf("Test %d (%s): Expected host '%s', got '%s'", i, test.input, test.host, actual.Host)
+		}
+		if actual.Port != test.port {
+			t.Errorf("Test %d (%s): Expected port '%s', got '%s'", i, test.input, test.port, actual.Port)
+		}
+		if actual.Path != test.path {
+			t.Errorf("Test %d (%s): Expected path '%s', got '%s'", i, test.input, test.path, actual.Path)
+		}
+	}
+}
+
+func TestAddressString(t *testing.T) {
+	for i, test := range []struct {
+		addr     Address
+		expected string
+	}{
+		{Address{Scheme: "http", Host: "host", Port: "1234", Path: "/path"}, "http://host:1234/path"},
+		{Address{Scheme: "", Host: "host", Port: "", Path: ""}, "http://host"},
+		{Address{Scheme: "", Host: "host", Port: "80", Path: ""}, "http://host"},
+		{Address{Scheme: "", Host: "host", Port: "443", Path: ""}, "https://host"},
+		{Address{Scheme: "https", Host: "host", Port: "443", Path: ""}, "https://host"},
+		{Address{Scheme: "https", Host: "host", Port: "", Path: ""}, "https://host"},
+		{Address{Scheme: "", Host: "host", Port: "80", Path: "/path"}, "http://host/path"},
+		{Address{Scheme: "http", Host: "", Port: "1234", Path: ""}, "http://:1234"},
+		{Address{Scheme: "", Host: "", Port: "", Path: ""}, ""},
+	} {
+		actual := test.addr.String()
+		if actual != test.expected {
+			t.Errorf("Test %d: expected '%s' but got '%s'", i, test.expected, actual)
+		}
+	}
+}
+
+func TestKeyNormalization(t *testing.T) {
+	testCases := []struct {
+		input  string
+		expect string
+	}{
+		{
+			input:  "http://host:1234/path",
+			expect: "http://host:1234/path",
+		},
+		{
+			input:  "HTTP://A/ABCDEF",
+			expect: "http://a/ABCDEF",
+		},
+		{
+			input:  "A/ABCDEF",
+			expect: "a/ABCDEF",
+		},
+		{
+			input:  "A:2015/Path",
+			expect: "a:2015/Path",
+		},
+		{
+			input:  ":80",
+			expect: ":80",
+		},
+		{
+			input:  ":443",
+			expect: ":443",
+		},
+		{
+			input:  ":1234",
+			expect: ":1234",
+		},
+		{
+			input:  "",
+			expect: "",
+		},
+		{
+			input:  ":",
+			expect: "",
+		},
+		{
+			input:  "[::]",
+			expect: "::",
+		},
+	}
+	for i, tc := range testCases {
+		addr, err := ParseAddress(tc.input)
+		if err != nil {
+			t.Errorf("Test %d: Parsing address '%s': %v", i, tc.input, err)
+			continue
+		}
+		expect := tc.expect
+		if !caseSensitivePath {
+			// every other part of the address should be lowercased when normalized,
+			// so simply lower-case the whole thing to do case-insensitive comparison
+			// of the path as well
+			expect = strings.ToLower(expect)
+		}
+		if actual := addr.Normalize().Key(); actual != expect {
+			t.Errorf("Test %d: Normalized key for address '%s' was '%s' but expected '%s'", i, tc.input, actual, expect)
+		}
+
+	}
+}

caddyconfig/httpcaddyfile/builtins.go

@@ -0,0 +1,273 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpcaddyfile
+
+import (
+	"encoding/json"
+	"fmt"
+	"html"
+	"net/http"
+	"reflect"
+
+	"github.com/caddyserver/caddy/v2/caddyconfig"
+	"github.com/caddyserver/caddy/v2/modules/caddyhttp"
+	"github.com/caddyserver/caddy/v2/modules/caddytls"
+)
+
+func init() {
+	RegisterDirective("bind", parseBind)
+	RegisterDirective("root", parseRoot)
+	RegisterDirective("tls", parseTLS)
+	RegisterHandlerDirective("redir", parseRedir)
+	RegisterHandlerDirective("respond", parseRespond)
+}
+
+func parseBind(h Helper) ([]ConfigValue, error) {
+	var lnHosts []string
+	for h.Next() {
+		lnHosts = append(lnHosts, h.RemainingArgs()...)
+	}
+	return h.NewBindAddresses(lnHosts), nil
+}
+
+func parseRoot(h Helper) ([]ConfigValue, error) {
+	if !h.Next() {
+		return nil, h.ArgErr()
+	}
+
+	matcherSet, ok, err := h.MatcherToken()
+	if err != nil {
+		return nil, err
+	}
+	if !ok {
+		// no matcher token; oops
+		h.Dispenser.Prev()
+	}
+
+	if !h.NextArg() {
+		return nil, h.ArgErr()
+	}
+	root := h.Val()
+	if h.NextArg() {
+		return nil, h.ArgErr()
+	}
+
+	varsHandler := caddyhttp.VarsMiddleware{"root": root}
+	route := caddyhttp.Route{
+		HandlersRaw: []json.RawMessage{
+			caddyconfig.JSONModuleObject(varsHandler, "handler", "vars", nil),
+		},
+	}
+	if matcherSet != nil {
+		route.MatcherSetsRaw = []map[string]json.RawMessage{matcherSet}
+	}
+
+	return h.NewVarsRoute(route), nil
+}
+
+func parseTLS(h Helper) ([]ConfigValue, error) {
+	var configVals []ConfigValue
+
+	cp := new(caddytls.ConnectionPolicy)
+	var fileLoader caddytls.FileLoader
+	var folderLoader caddytls.FolderLoader
+	var mgr caddytls.ACMEManagerMaker
+	var off bool
+
+	// fill in global defaults, if configured
+	if email := h.Option("email"); email != nil {
+		mgr.Email = email.(string)
+	}
+	if acmeCA := h.Option("acme_ca"); acmeCA != nil {
+		mgr.CA = acmeCA.(string)
+	}
+
+	for h.Next() {
+		// file certificate loader
+		firstLine := h.RemainingArgs()
+		switch len(firstLine) {
+		case 0:
+		case 1:
+			if firstLine[0] == "off" {
+				off = true
+			} else {
+				mgr.Email = firstLine[0]
+			}
+		case 2:
+			fileLoader = append(fileLoader, caddytls.CertKeyFilePair{
+				Certificate: firstLine[0],
+				Key:         firstLine[1],
+				// TODO: add tags, for enterprise module's certificate selection
+			})
+		default:
+			return nil, h.ArgErr()
+		}
+
+		var hasBlock bool
+		for h.NextBlock(0) {
+			hasBlock = true
+
+			switch h.Val() {
+			// connection policy
+			case "protocols":
+				args := h.RemainingArgs()
+				if len(args) == 0 {
+					return nil, h.SyntaxErr("one or two protocols")
+				}
+				if len(args) > 0 {
+					if _, ok := caddytls.SupportedProtocols[args[0]]; !ok {
+						return nil, h.Errf("Wrong protocol name or protocol not supported: '%s'", args[0])
+					}
+					cp.ProtocolMin = args[0]
+				}
+				if len(args) > 1 {
+					if _, ok := caddytls.SupportedProtocols[args[1]]; !ok {
+						return nil, h.Errf("Wrong protocol name or protocol not supported: '%s'", args[1])
+					}
+					cp.ProtocolMax = args[1]
+				}
+			case "ciphers":
+				for h.NextArg() {
+					if _, ok := caddytls.SupportedCipherSuites[h.Val()]; !ok {
+						return nil, h.Errf("Wrong cipher suite name or cipher suite not supported: '%s'", h.Val())
+					}
+					cp.CipherSuites = append(cp.CipherSuites, h.Val())
+				}
+			case "curves":
+				for h.NextArg() {
+					if _, ok := caddytls.SupportedCurves[h.Val()]; !ok {
+						return nil, h.Errf("Wrong curve name or curve not supported: '%s'", h.Val())
+					}
+					cp.Curves = append(cp.Curves, h.Val())
+				}
+			case "alpn":
+				args := h.RemainingArgs()
+				if len(args) == 0 {
+					return nil, h.ArgErr()
+				}
+				cp.ALPN = args
+
+			// certificate folder loader
+			case "load":
+				folderLoader = append(folderLoader, h.RemainingArgs()...)
+
+			// automation policy
+			case "ca":
+				arg := h.RemainingArgs()
+				if len(arg) != 1 {
+					return nil, h.ArgErr()
+				}
+				mgr.CA = arg[0]
+
+			default:
+				return nil, h.Errf("unknown subdirective: %s", h.Val())
+			}
+		}
+
+		// a naked tls directive is not allowed
+		if len(firstLine) == 0 && !hasBlock {
+			return nil, h.ArgErr()
+		}
+	}
+
+	// connection policy
+	configVals = append(configVals, ConfigValue{
+		Class: "tls.connection_policy",
+		Value: cp,
+	})
+
+	// certificate loaders
+	if len(fileLoader) > 0 {
+		configVals = append(configVals, ConfigValue{
+			Class: "tls.certificate_loader",
+			Value: fileLoader,
+		})
+	}
+	if len(folderLoader) > 0 {
+		configVals = append(configVals, ConfigValue{
+			Class: "tls.certificate_loader",
+			Value: folderLoader,
+		})
+	}
+
+	// automation policy
+	if off {
+		configVals = append(configVals, ConfigValue{
+			Class: "tls.off",
+			Value: true,
+		})
+	} else if !reflect.DeepEqual(mgr, caddytls.ACMEManagerMaker{}) {
+		configVals = append(configVals, ConfigValue{
+			Class: "tls.automation_manager",
+			Value: mgr,
+		})
+	}
+
+	return configVals, nil
+}
+
+func parseRedir(h Helper) (caddyhttp.MiddlewareHandler, error) {
+	if !h.Next() {
+		return nil, h.ArgErr()
+	}
+
+	if !h.NextArg() {
+		return nil, h.ArgErr()
+	}
+	to := h.Val()
+
+	var code string
+	if h.NextArg() {
+		code = h.Val()
+	}
+	if code == "permanent" {
+		code = "301"
+	}
+	if code == "temporary" || code == "" {
+		code = "307"
+	}
+	var body string
+	if code == "meta" {
+		// Script tag comes first since that will better imitate a redirect in the browser's
+		// history, but the meta tag is a fallback for most non-JS clients.
+		const metaRedir = `<!DOCTYPE html>
+<html>
+	<head>
+		<title>Redirecting...</title>
+		<script>window.location.replace("%s");</script>
+		<meta http-equiv="refresh" content="0; URL='%s'">
+	</head>
+	<body>Redirecting to <a href="%s">%s</a>...</body>
+</html>
+`
+		safeTo := html.EscapeString(to)
+		body = fmt.Sprintf(metaRedir, safeTo, safeTo, safeTo, safeTo)
+	}
+
+	return caddyhttp.StaticResponse{
+		StatusCode: caddyhttp.WeakString(code),
+		Headers:    http.Header{"Location": []string{to}},
+		Body:       body,
+	}, nil
+}
+
+func parseRespond(h Helper) (caddyhttp.MiddlewareHandler, error) {
+	sr := new(caddyhttp.StaticResponse)
+	err := sr.UnmarshalCaddyfile(h.Dispenser)
+	if err != nil {
+		return nil, err
+	}
+	return sr, nil
+}

caddyconfig/httpcaddyfile/directives.go

@@ -0,0 +1,210 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpcaddyfile
+
+import (
+	"encoding/json"
+
+	"github.com/caddyserver/caddy/v2"
+	"github.com/caddyserver/caddy/v2/caddyconfig"
+	"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
+	"github.com/caddyserver/caddy/v2/modules/caddyhttp"
+)
+
+// defaultDirectiveOrder specifies the order
+// to apply directives in HTTP routes.
+var defaultDirectiveOrder = []string{
+	"rewrite",
+	"try_files",
+	"basicauth",
+	"headers",
+	"request_header",
+	"encode",
+	"templates",
+	"redir",
+	"respond",
+	"reverse_proxy",
+	"php_fastcgi",
+	"file_server",
+}
+
+// RegisterDirective registers a unique directive dir with an
+// associated unmarshaling (setup) function. When directive dir
+// is encountered in a Caddyfile, setupFunc will be called to
+// unmarshal its tokens.
+func RegisterDirective(dir string, setupFunc UnmarshalFunc) {
+	if _, ok := registeredDirectives[dir]; ok {
+		panic("directive " + dir + " already registered")
+	}
+	registeredDirectives[dir] = setupFunc
+}
+
+// RegisterHandlerDirective is like RegisterDirective, but for
+// directives which specifically output only an HTTP handler.
+func RegisterHandlerDirective(dir string, setupFunc UnmarshalHandlerFunc) {
+	RegisterDirective(dir, func(h Helper) ([]ConfigValue, error) {
+		if !h.Next() {
+			return nil, h.ArgErr()
+		}
+
+		matcherSet, ok, err := h.MatcherToken()
+		if err != nil {
+			return nil, err
+		}
+		if ok {
+			h.Dispenser.Delete() // strip matcher token
+		}
+
+		h.Dispenser.Reset() // pretend this lookahead never happened
+		val, err := setupFunc(h)
+		if err != nil {
+			return nil, err
+		}
+
+		return h.NewRoute(matcherSet, val), nil
+	})
+}
+
+// Helper is a type which helps setup a value from
+// Caddyfile tokens.
+type Helper struct {
+	*caddyfile.Dispenser
+	options     map[string]interface{}
+	warnings    *[]caddyconfig.Warning
+	matcherDefs map[string]map[string]json.RawMessage
+	parentBlock caddyfile.ServerBlock
+}
+
+// Option gets the option keyed by name.
+func (h Helper) Option(name string) interface{} {
+	return h.options[name]
+}
+
+// Caddyfiles returns the list of config files from
+// which tokens in the current server block were loaded.
+func (h Helper) Caddyfiles() []string {
+	// first obtain set of names of files involved
+	// in this server block, without duplicates
+	files := make(map[string]struct{})
+	for _, segment := range h.parentBlock.Segments {
+		for _, token := range segment {
+			files[token.File] = struct{}{}
+		}
+	}
+	// then convert the set into a slice
+	filesSlice := make([]string, 0, len(files))
+	for file := range files {
+		filesSlice = append(filesSlice, file)
+	}
+	return filesSlice
+}
+
+// JSON converts val into JSON. Any errors are added to warnings.
+func (h Helper) JSON(val interface{}, warnings *[]caddyconfig.Warning) json.RawMessage {
+	return caddyconfig.JSON(val, h.warnings)
+}
+
+// MatcherToken assumes the current token is (possibly) a matcher, and
+// if so, returns the matcher set along with a true value. If the current
+// token is not a matcher, nil and false is returned. Note that a true
+// value may be returned with a nil matcher set if it is a catch-all.
+func (h Helper) MatcherToken() (map[string]json.RawMessage, bool, error) {
+	if !h.NextArg() {
+		return nil, false, nil
+	}
+	return matcherSetFromMatcherToken(h.Dispenser.Token(), h.matcherDefs, h.warnings)
+}
+
+// NewRoute returns config values relevant to creating a new HTTP route.
+func (h Helper) NewRoute(matcherSet map[string]json.RawMessage,
+	handler caddyhttp.MiddlewareHandler) []ConfigValue {
+	mod, err := caddy.GetModule(caddy.GetModuleName(handler))
+	if err != nil {
+		// TODO: append to warnings
+	}
+	var matcherSetsRaw []map[string]json.RawMessage
+	if matcherSet != nil {
+		matcherSetsRaw = append(matcherSetsRaw, matcherSet)
+	}
+	return []ConfigValue{
+		{
+			Class: "route",
+			Value: caddyhttp.Route{
+				MatcherSetsRaw: matcherSetsRaw,
+				HandlersRaw:    []json.RawMessage{caddyconfig.JSONModuleObject(handler, "handler", mod.ID(), h.warnings)},
+			},
+		},
+	}
+}
+
+// NewBindAddresses returns config values relevant to adding
+// listener bind addresses to the config.
+func (h Helper) NewBindAddresses(addrs []string) []ConfigValue {
+	return []ConfigValue{{Class: "bind", Value: addrs}}
+}
+
+// NewVarsRoute returns config values relevant to adding a
+// "vars" wrapper route to the config.
+func (h Helper) NewVarsRoute(route caddyhttp.Route) []ConfigValue {
+	return []ConfigValue{{Class: "var", Value: route}}
+}
+
+// ConfigValue represents a value to be added to the final
+// configuration, or a value to be consulted when building
+// the final configuration.
+type ConfigValue struct {
+	// The kind of value this is. As the config is
+	// being built, the adapter will look in the
+	// "pile" for values belonging to a certain
+	// class when it is setting up a certain part
+	// of the config. The associated value will be
+	// type-asserted and placed accordingly.
+	Class string
+
+	// The value to be used when building the config.
+	// Generally its type is associated with the
+	// name of the Class.
+	Value interface{}
+
+	directive string
+}
+
+// serverBlock pairs a Caddyfile server block
+// with a "pile" of config values, keyed by class
+// name.
+type serverBlock struct {
+	block caddyfile.ServerBlock
+	pile  map[string][]ConfigValue // config values obtained from directives
+}
+
+type (
+	// UnmarshalFunc is a function which can unmarshal Caddyfile
+	// tokens into zero or more config values using a Helper type.
+	// These are passed in a call to RegisterDirective.
+	UnmarshalFunc func(h Helper) ([]ConfigValue, error)
+
+	// UnmarshalHandlerFunc is like UnmarshalFunc, except the
+	// output of the unmarshaling is an HTTP handler. This
+	// function does not need to deal with HTTP request matching
+	// which is abstracted away. Since writing HTTP handlers
+	// with Caddyfile support is very common, this is a more
+	// convenient way to add a handler to the chain since a lot
+	// of the details common to HTTP handlers are taken care of
+	// for you. These are passed to a call to
+	// RegisterHandlerDirective.
+	UnmarshalHandlerFunc func(h Helper) (caddyhttp.MiddlewareHandler, error)
+)
+
+var registeredDirectives = make(map[string]UnmarshalFunc)

caddyconfig/httpcaddyfile/httptype.go

@@ -0,0 +1,633 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpcaddyfile
+
+import (
+	"encoding/json"
+	"fmt"
+	"reflect"
+	"sort"
+	"strings"
+
+	"github.com/caddyserver/caddy/v2"
+	"github.com/caddyserver/caddy/v2/caddyconfig"
+	"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
+	"github.com/caddyserver/caddy/v2/modules/caddyhttp"
+	"github.com/caddyserver/caddy/v2/modules/caddytls"
+	"github.com/mholt/certmagic"
+)
+
+func init() {
+	caddyconfig.RegisterAdapter("caddyfile", caddyfile.Adapter{ServerType: ServerType{}})
+}
+
+// ServerType can set up a config from an HTTP Caddyfile.
+type ServerType struct {
+}
+
+// Setup makes a config from the tokens.
+func (st ServerType) Setup(originalServerBlocks []caddyfile.ServerBlock,
+	options map[string]interface{}) (*caddy.Config, []caddyconfig.Warning, error) {
+	var warnings []caddyconfig.Warning
+
+	var serverBlocks []serverBlock
+	for _, sblock := range originalServerBlocks {
+		serverBlocks = append(serverBlocks, serverBlock{
+			block: sblock,
+			pile:  make(map[string][]ConfigValue),
+		})
+	}
+
+	// global configuration
+	if len(serverBlocks) > 0 && len(serverBlocks[0].block.Keys) == 0 {
+		sb := serverBlocks[0]
+		for _, segment := range sb.block.Segments {
+			dir := segment.Directive()
+			var val interface{}
+			var err error
+			disp := caddyfile.NewDispenser(segment)
+			// TODO: make this switch into a map
+			switch dir {
+			case "http_port":
+				val, err = parseOptHTTPPort(disp)
+			case "https_port":
+				val, err = parseOptHTTPSPort(disp)
+			case "handler_order":
+				val, err = parseOptHandlerOrder(disp)
+			case "experimental_http3":
+				val, err = parseOptExperimentalHTTP3(disp)
+			case "storage":
+				val, err = parseOptStorage(disp)
+			case "acme_ca":
+				val, err = parseOptACMECA(disp)
+			case "email":
+				val, err = parseOptEmail(disp)
+			default:
+				return nil, warnings, fmt.Errorf("unrecognized parameter name: %s", dir)
+			}
+			if err != nil {
+				return nil, warnings, fmt.Errorf("%s: %v", dir, err)
+			}
+			options[dir] = val
+		}
+		serverBlocks = serverBlocks[1:]
+	}
+
+	for _, sb := range serverBlocks {
+		// replace shorthand placeholders (which are
+		// convenient when writing a Caddyfile) with
+		// their actual placeholder identifiers or
+		// variable names
+		replacer := strings.NewReplacer(
+			"{uri}", "{http.request.uri}",
+			"{path}", "{http.request.uri.path}",
+			"{host}", "{http.request.host}",
+			"{hostport}", "{http.request.hostport}",
+			"{method}", "{http.request.method}",
+			"{scheme}", "{http.request.scheme}",
+			"{file}", "{http.request.uri.path.file}",
+			"{dir}", "{http.request.uri.path.dir}",
+			"{query}", "{http.request.uri.query_string}",
+		)
+		for _, segment := range sb.block.Segments {
+			for i := 0; i < len(segment); i++ {
+				segment[i].Text = replacer.Replace(segment[i].Text)
+			}
+		}
+
+		if len(sb.block.Keys) == 0 {
+			return nil, warnings, fmt.Errorf("server block without any key is global configuration, and if used, it must be first")
+		}
+
+		// extract matcher definitions
+		d := sb.block.DispenseDirective("matcher")
+		matcherDefs, err := parseMatcherDefinitions(d)
+		if err != nil {
+			return nil, warnings, err
+		}
+
+		for _, segment := range sb.block.Segments {
+			dir := segment.Directive()
+			if dir == "matcher" {
+				// TODO: This is a special case because we pre-processed it; handle this better
+				continue
+			}
+			if dirFunc, ok := registeredDirectives[dir]; ok {
+				results, err := dirFunc(Helper{
+					Dispenser:   caddyfile.NewDispenser(segment),
+					options:     options,
+					warnings:    &warnings,
+					matcherDefs: matcherDefs,
+					parentBlock: sb.block,
+				})
+				if err != nil {
+					return nil, warnings, fmt.Errorf("parsing caddyfile tokens for '%s': %v", dir, err)
+				}
+				for _, result := range results {
+					result.directive = dir
+					sb.pile[result.Class] = append(sb.pile[result.Class], result)
+				}
+			} else {
+				tkn := segment[0]
+				return nil, warnings, fmt.Errorf("%s:%d: unrecognized directive: %s", tkn.File, tkn.Line, dir)
+			}
+		}
+	}
+
+	// map
+	sbmap, err := st.mapAddressToServerBlocks(serverBlocks)
+	if err != nil {
+		return nil, warnings, err
+	}
+
+	// reduce
+	pairings := st.consolidateAddrMappings(sbmap)
+
+	// each pairing of listener addresses to list of server
+	// blocks is basically a server definition
+	servers, err := st.serversFromPairings(pairings, options, &warnings)
+	if err != nil {
+		return nil, warnings, err
+	}
+
+	// now that each server is configured, make the HTTP app
+	httpApp := caddyhttp.App{
+		HTTPPort:  tryInt(options["http_port"], &warnings),
+		HTTPSPort: tryInt(options["https_port"], &warnings),
+		Servers:   servers,
+	}
+
+	// now for the TLS app! (TODO: refactor into own func)
+	tlsApp := caddytls.TLS{Certificates: make(map[string]json.RawMessage)}
+	for _, p := range pairings {
+		for i, sblock := range p.serverBlocks {
+			// tls automation policies
+			if mmVals, ok := sblock.pile["tls.automation_manager"]; ok {
+				for _, mmVal := range mmVals {
+					mm := mmVal.Value.(caddytls.ManagerMaker)
+					sblockHosts, err := st.autoHTTPSHosts(sblock)
+					if err != nil {
+						return nil, warnings, err
+					}
+					if len(sblockHosts) > 0 {
+						if tlsApp.Automation == nil {
+							tlsApp.Automation = new(caddytls.AutomationConfig)
+						}
+						tlsApp.Automation.Policies = append(tlsApp.Automation.Policies, caddytls.AutomationPolicy{
+							Hosts:         sblockHosts,
+							ManagementRaw: caddyconfig.JSONModuleObject(mm, "module", mm.(caddy.Module).CaddyModule().ID(), &warnings),
+						})
+					} else {
+						warnings = append(warnings, caddyconfig.Warning{
+							Message: fmt.Sprintf("Server block %d %v has no names that qualify for automatic HTTPS, so no TLS automation policy will be added.", i, sblock.block.Keys),
+						})
+					}
+				}
+			}
+
+			// tls certificate loaders
+			if clVals, ok := sblock.pile["tls.certificate_loader"]; ok {
+				for _, clVal := range clVals {
+					loader := clVal.Value.(caddytls.CertificateLoader)
+					loaderName := caddy.GetModuleID(loader)
+					tlsApp.Certificates[loaderName] = caddyconfig.JSON(loader, &warnings)
+				}
+			}
+		}
+	}
+	// if global ACME CA or email were set, append a catch-all automation
+	// policy that ensures they will be used if no tls directive was used
+	acmeCA, hasACMECA := options["acme_ca"]
+	email, hasEmail := options["email"]
+	if hasACMECA || hasEmail {
+		if tlsApp.Automation == nil {
+			tlsApp.Automation = new(caddytls.AutomationConfig)
+		}
+		tlsApp.Automation.Policies = append(tlsApp.Automation.Policies, caddytls.AutomationPolicy{
+			ManagementRaw: caddyconfig.JSONModuleObject(caddytls.ACMEManagerMaker{
+				CA:    acmeCA.(string),
+				Email: email.(string),
+			}, "module", "acme", &warnings),
+		})
+	}
+	if tlsApp.Automation != nil {
+		// consolidate automation policies that are the exact same
+		tlsApp.Automation.Policies = consolidateAutomationPolicies(tlsApp.Automation.Policies)
+	}
+
+	// if experimental HTTP/3 is enabled, enable it on each server
+	if enableH3, ok := options["experimental_http3"].(bool); ok && enableH3 {
+		for _, srv := range httpApp.Servers {
+			srv.ExperimentalHTTP3 = true
+		}
+	}
+
+	// annnd the top-level config, then we're done!
+	cfg := &caddy.Config{AppsRaw: make(map[string]json.RawMessage)}
+	if !reflect.DeepEqual(httpApp, caddyhttp.App{}) {
+		cfg.AppsRaw["http"] = caddyconfig.JSON(httpApp, &warnings)
+	}
+	if !reflect.DeepEqual(tlsApp, caddytls.TLS{Certificates: make(map[string]json.RawMessage)}) {
+		cfg.AppsRaw["tls"] = caddyconfig.JSON(tlsApp, &warnings)
+	}
+	if storageCvtr, ok := options["storage"].(caddy.StorageConverter); ok {
+		cfg.StorageRaw = caddyconfig.JSONModuleObject(storageCvtr,
+			"module",
+			storageCvtr.(caddy.Module).CaddyModule().ID(),
+			&warnings)
+	}
+
+	return cfg, warnings, nil
+}
+
+// hostsFromServerBlockKeys returns a list of all the
+// hostnames found in the keys of the server block sb.
+// The list may not be in a consistent order.
+func (st *ServerType) hostsFromServerBlockKeys(sb caddyfile.ServerBlock) ([]string, error) {
+	// first get each unique hostname
+	hostMap := make(map[string]struct{})
+	for _, sblockKey := range sb.Keys {
+		addr, err := ParseAddress(sblockKey)
+		if err != nil {
+			return nil, fmt.Errorf("parsing server block key: %v", err)
+		}
+		addr = addr.Normalize()
+		hostMap[addr.Host] = struct{}{}
+	}
+
+	// convert map to slice
+	sblockHosts := make([]string, 0, len(hostMap))
+	for host := range hostMap {
+		sblockHosts = append(sblockHosts, host)
+	}
+
+	return sblockHosts, nil
+}
+
+// serversFromPairings creates the servers for each pairing of addresses
+// to server blocks. Each pairing is essentially a server definition.
+func (st *ServerType) serversFromPairings(
+	pairings []sbAddrAssociation,
+	options map[string]interface{},
+	warnings *[]caddyconfig.Warning,
+) (map[string]*caddyhttp.Server, error) {
+	servers := make(map[string]*caddyhttp.Server)
+
+	for i, p := range pairings {
+		srv := &caddyhttp.Server{
+			Listen: p.addresses,
+		}
+
+		for _, sblock := range p.serverBlocks {
+			matcherSetsEnc, err := st.compileEncodedMatcherSets(sblock.block)
+			if err != nil {
+				return nil, fmt.Errorf("server block %v: compiling matcher sets: %v", sblock.block.Keys, err)
+			}
+
+			// if there are user-defined variables, then siteVarSubroute will
+			// wrap the handlerSubroute; otherwise handlerSubroute will be the
+			// site's primary subroute.
+			siteVarSubroute, handlerSubroute := new(caddyhttp.Subroute), new(caddyhttp.Subroute)
+
+			// tls: connection policies and toggle auto HTTPS
+
+			autoHTTPSQualifiedHosts, err := st.autoHTTPSHosts(sblock)
+			if err != nil {
+				return nil, err
+			}
+			if _, ok := sblock.pile["tls.off"]; ok && len(autoHTTPSQualifiedHosts) > 0 {
+				// tls off: disable TLS (and automatic HTTPS) for server block's names
+				if srv.AutoHTTPS == nil {
+					srv.AutoHTTPS = new(caddyhttp.AutoHTTPSConfig)
+				}
+				srv.AutoHTTPS.Skip = append(srv.AutoHTTPS.Skip, autoHTTPSQualifiedHosts...)
+			} else if cpVals, ok := sblock.pile["tls.connection_policy"]; ok {
+				// tls connection policies
+				for _, cpVal := range cpVals {
+					cp := cpVal.Value.(*caddytls.ConnectionPolicy)
+					// only create if there is a non-empty policy
+					if !reflect.DeepEqual(cp, new(caddytls.ConnectionPolicy)) {
+						// make sure the policy covers all hostnames from the block
+						hosts, err := st.hostsFromServerBlockKeys(sblock.block)
+						if err != nil {
+							return nil, err
+						}
+
+						// TODO: are matchers needed if every hostname of the config is matched?
+						cp.Matchers = map[string]json.RawMessage{
+							"sni": caddyconfig.JSON(hosts, warnings), // make sure to match all hosts, not just auto-HTTPS-qualified ones
+						}
+						srv.TLSConnPolicies = append(srv.TLSConnPolicies, cp)
+					}
+				}
+				// TODO: consolidate equal conn policies
+			}
+
+			// vars: special routes that will have to wrap the normal handlers
+			// so that these variables can be used across their matchers too
+			for _, cfgVal := range sblock.pile["var"] {
+				siteVarSubroute.Routes = append(siteVarSubroute.Routes, cfgVal.Value.(caddyhttp.Route))
+			}
+
+			// set up each handler directive - the order of the handlers
+			// as they are added to the routes depends on user preference
+			dirRoutes := sblock.pile["route"]
+			handlerOrder, ok := options["handler_order"].([]string)
+			if !ok {
+				handlerOrder = defaultDirectiveOrder
+			}
+			if len(handlerOrder) == 1 && handlerOrder[0] == "appearance" {
+				handlerOrder = nil
+			}
+			if handlerOrder != nil {
+				dirPositions := make(map[string]int)
+				for i, dir := range handlerOrder {
+					dirPositions[dir] = i
+				}
+				sort.SliceStable(dirRoutes, func(i, j int) bool {
+					iDir, jDir := dirRoutes[i].directive, dirRoutes[j].directive
+					return dirPositions[iDir] < dirPositions[jDir]
+				})
+			}
+			for _, r := range dirRoutes {
+				handlerSubroute.Routes = append(handlerSubroute.Routes, r.Value.(caddyhttp.Route))
+			}
+
+			// the route that contains the site's handlers will
+			// be assumed to be the sub-route for this site...
+			siteSubroute := handlerSubroute
+
+			// ... unless, of course, there are variables that might
+			// be used by the site's matchers or handlers, in which
+			// case we need to nest the handlers in a sub-sub-route,
+			// and the variables go in the sub-route so the variables
+			// get evaluated first
+			if len(siteVarSubroute.Routes) > 0 {
+				subSubRoute := caddyhttp.Subroute{Routes: siteSubroute.Routes}
+				siteSubroute.Routes = append(
+					siteVarSubroute.Routes,
+					caddyhttp.Route{
+						HandlersRaw: []json.RawMessage{
+							caddyconfig.JSONModuleObject(subSubRoute, "handler", "subroute", warnings),
+						},
+					},
+				)
+			}
+
+			siteSubroute.Routes = consolidateRoutes(siteSubroute.Routes)
+
+			srv.Routes = append(srv.Routes, caddyhttp.Route{
+				MatcherSetsRaw: matcherSetsEnc,
+				HandlersRaw: []json.RawMessage{
+					caddyconfig.JSONModuleObject(siteSubroute, "handler", "subroute", warnings),
+				},
+			})
+		}
+
+		srv.Routes = consolidateRoutes(srv.Routes)
+
+		servers[fmt.Sprintf("srv%d", i)] = srv
+	}
+
+	return servers, nil
+}
+
+func (st ServerType) autoHTTPSHosts(sb serverBlock) ([]string, error) {
+	// get the hosts for this server block...
+	hosts, err := st.hostsFromServerBlockKeys(sb.block)
+	if err != nil {
+		return nil, err
+	}
+	// ...and of those, which ones qualify for auto HTTPS
+	var autoHTTPSQualifiedHosts []string
+	for _, h := range hosts {
+		if certmagic.HostQualifies(h) {
+			autoHTTPSQualifiedHosts = append(autoHTTPSQualifiedHosts, h)
+		}
+	}
+	return autoHTTPSQualifiedHosts, nil
+}
+
+// consolidateRoutes combines routes with the same properties
+// (same matchers, same Terminal and Group settings) for a
+// cleaner overall output.
+func consolidateRoutes(routes caddyhttp.RouteList) caddyhttp.RouteList {
+	for i := 0; i < len(routes)-1; i++ {
+		if reflect.DeepEqual(routes[i].MatcherSetsRaw, routes[i+1].MatcherSetsRaw) &&
+			routes[i].Terminal == routes[i+1].Terminal &&
+			routes[i].Group == routes[i+1].Group {
+			// keep the handlers in the same order, then splice out repetitive route
+			routes[i].HandlersRaw = append(routes[i].HandlersRaw, routes[i+1].HandlersRaw...)
+			routes = append(routes[:i+1], routes[i+2:]...)
+			i--
+		}
+	}
+	return routes
+}
+
+// consolidateAutomationPolicies combines automation policies that are the same,
+// for a cleaner overall output.
+func consolidateAutomationPolicies(aps []caddytls.AutomationPolicy) []caddytls.AutomationPolicy {
+	for i := 0; i < len(aps); i++ {
+		for j := 0; j < len(aps); j++ {
+			if j == i {
+				continue
+			}
+			if reflect.DeepEqual(aps[i].ManagementRaw, aps[j].ManagementRaw) {
+				aps[i].Hosts = append(aps[i].Hosts, aps[j].Hosts...)
+				aps = append(aps[:j], aps[j+1:]...)
+				i--
+				break
+			}
+		}
+	}
+	return aps
+}
+
+func matcherSetFromMatcherToken(
+	tkn caddyfile.Token,
+	matcherDefs map[string]map[string]json.RawMessage,
+	warnings *[]caddyconfig.Warning,
+) (map[string]json.RawMessage, bool, error) {
+	// matcher tokens can be wildcards, simple path matchers,
+	// or refer to a pre-defined matcher by some name
+	if tkn.Text == "*" {
+		// match all requests == no matchers, so nothing to do
+		return nil, true, nil
+	} else if strings.HasPrefix(tkn.Text, "/") {
+		// convenient way to specify a single path match
+		return map[string]json.RawMessage{
+			"path": caddyconfig.JSON(caddyhttp.MatchPath{tkn.Text}, warnings),
+		}, true, nil
+	} else if strings.HasPrefix(tkn.Text, "match:") {
+		// pre-defined matcher
+		matcherName := strings.TrimPrefix(tkn.Text, "match:")
+		m, ok := matcherDefs[matcherName]
+		if !ok {
+			return nil, false, fmt.Errorf("unrecognized matcher name: %+v", matcherName)
+		}
+		return m, true, nil
+	}
+
+	return nil, false, nil
+}
+
+func (st *ServerType) compileEncodedMatcherSets(sblock caddyfile.ServerBlock) ([]map[string]json.RawMessage, error) {
+	type hostPathPair struct {
+		hostm caddyhttp.MatchHost
+		pathm caddyhttp.MatchPath
+	}
+
+	// keep routes with common host and path matchers together
+	var matcherPairs []*hostPathPair
+
+	for _, key := range sblock.Keys {
+		addr, err := ParseAddress(key)
+		if err != nil {
+			return nil, fmt.Errorf("server block %v: parsing and standardizing address '%s': %v", sblock.Keys, key, err)
+		}
+		addr = addr.Normalize()
+
+		// choose a matcher pair that should be shared by this
+		// server block; if none exists yet, create one
+		var chosenMatcherPair *hostPathPair
+		for _, mp := range matcherPairs {
+			if (len(mp.pathm) == 0 && addr.Path == "") ||
+				(len(mp.pathm) == 1 && mp.pathm[0] == addr.Path) {
+				chosenMatcherPair = mp
+				break
+			}
+		}
+		if chosenMatcherPair == nil {
+			chosenMatcherPair = new(hostPathPair)
+			if addr.Path != "" {
+				chosenMatcherPair.pathm = []string{addr.Path}
+			}
+			matcherPairs = append(matcherPairs, chosenMatcherPair)
+		}
+
+		// add this server block's keys to the matcher
+		// pair if it doesn't already exist
+		if addr.Host != "" {
+			var found bool
+			for _, h := range chosenMatcherPair.hostm {
+				if h == addr.Host {
+					found = true
+					break
+				}
+			}
+			if !found {
+				chosenMatcherPair.hostm = append(chosenMatcherPair.hostm, addr.Host)
+			}
+		}
+	}
+
+	// iterate each pairing of host and path matchers and
+	// put them into a map for JSON encoding
+	var matcherSets []map[string]caddyhttp.RequestMatcher
+	for _, mp := range matcherPairs {
+		matcherSet := make(map[string]caddyhttp.RequestMatcher)
+		if len(mp.hostm) > 0 {
+			matcherSet["host"] = mp.hostm
+		}
+		if len(mp.pathm) > 0 {
+			matcherSet["path"] = mp.pathm
+		}
+		if len(matcherSet) > 0 {
+			matcherSets = append(matcherSets, matcherSet)
+		}
+	}
+
+	// finally, encode each of the matcher sets
+	var matcherSetsEnc []map[string]json.RawMessage
+	for _, ms := range matcherSets {
+		msEncoded, err := encodeMatcherSet(ms)
+		if err != nil {
+			return nil, fmt.Errorf("server block %v: %v", sblock.Keys, err)
+		}
+		matcherSetsEnc = append(matcherSetsEnc, msEncoded)
+	}
+
+	return matcherSetsEnc, nil
+}
+
+func parseMatcherDefinitions(d *caddyfile.Dispenser) (map[string]map[string]json.RawMessage, error) {
+	matchers := make(map[string]map[string]json.RawMessage)
+	for d.Next() {
+		definitionName := d.Val()
+		for nesting := d.Nesting(); d.NextBlock(nesting); {
+			matcherName := d.Val()
+			mod, err := caddy.GetModule("http.matchers." + matcherName)
+			if err != nil {
+				return nil, fmt.Errorf("getting matcher module '%s': %v", matcherName, err)
+			}
+			unm, ok := mod.New().(caddyfile.Unmarshaler)
+			if !ok {
+				return nil, fmt.Errorf("matcher module '%s' is not a Caddyfile unmarshaler", matcherName)
+			}
+			err = unm.UnmarshalCaddyfile(d.NewFromNextTokens())
+			if err != nil {
+				return nil, err
+			}
+			rm, ok := unm.(caddyhttp.RequestMatcher)
+			if !ok {
+				return nil, fmt.Errorf("matcher module '%s' is not a request matcher", matcherName)
+			}
+			if _, ok := matchers[definitionName]; !ok {
+				matchers[definitionName] = make(map[string]json.RawMessage)
+			}
+			matchers[definitionName][matcherName] = caddyconfig.JSON(rm, nil)
+		}
+	}
+	return matchers, nil
+}
+
+func encodeMatcherSet(matchers map[string]caddyhttp.RequestMatcher) (map[string]json.RawMessage, error) {
+	msEncoded := make(map[string]json.RawMessage)
+	for matcherName, val := range matchers {
+		jsonBytes, err := json.Marshal(val)
+		if err != nil {
+			return nil, fmt.Errorf("marshaling matcher set %#v: %v", matchers, err)
+		}
+		msEncoded[matcherName] = jsonBytes
+	}
+	return msEncoded, nil
+}
+
+// tryInt tries to convert val to an integer. If it fails,
+// it downgrades the error to a warning and returns 0.
+func tryInt(val interface{}, warnings *[]caddyconfig.Warning) int {
+	intVal, ok := val.(int)
+	if val != nil && !ok && warnings != nil {
+		*warnings = append(*warnings, caddyconfig.Warning{Message: "not an integer type"})
+	}
+	return intVal
+}
+
+type matcherSetAndTokens struct {
+	matcherSet map[string]json.RawMessage
+	tokens     []caddyfile.Token
+}
+
+// sbAddrAssocation is a mapping from a list of
+// addresses to a list of server blocks that are
+// served on those addresses.
+type sbAddrAssociation struct {
+	addresses    []string
+	serverBlocks []serverBlock
+}
+
+// Interface guard
+var _ caddyfile.ServerType = (*ServerType)(nil)

caddyconfig/httpcaddyfile/options.go

@@ -0,0 +1,134 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package httpcaddyfile
+
+import (
+	"fmt"
+	"strconv"
+
+	"github.com/caddyserver/caddy/v2"
+	"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
+)
+
+func parseOptHTTPPort(d *caddyfile.Dispenser) (int, error) {
+	var httpPort int
+	for d.Next() {
+		var httpPortStr string
+		if !d.AllArgs(&httpPortStr) {
+			return 0, d.ArgErr()
+		}
+		var err error
+		httpPort, err = strconv.Atoi(httpPortStr)
+		if err != nil {
+			return 0, d.Errf("converting port '%s' to integer value: %v", httpPortStr, err)
+		}
+	}
+	return httpPort, nil
+}
+
+func parseOptHTTPSPort(d *caddyfile.Dispenser) (int, error) {
+	var httpsPort int
+	for d.Next() {
+		var httpsPortStr string
+		if !d.AllArgs(&httpsPortStr) {
+			return 0, d.ArgErr()
+		}
+		var err error
+		httpsPort, err = strconv.Atoi(httpsPortStr)
+		if err != nil {
+			return 0, d.Errf("converting port '%s' to integer value: %v", httpsPortStr, err)
+		}
+	}
+	return httpsPort, nil
+}
+
+func parseOptExperimentalHTTP3(d *caddyfile.Dispenser) (bool, error) {
+	return true, nil
+}
+
+func parseOptHandlerOrder(d *caddyfile.Dispenser) ([]string, error) {
+	if !d.Next() {
+		return nil, d.ArgErr()
+	}
+	order := d.RemainingArgs()
+	if len(order) == 1 && order[0] == "appearance" {
+		return []string{"appearance"}, nil
+	}
+	if len(order) > 0 && d.NextBlock(0) {
+		return nil, d.Err("cannot open block if there are arguments")
+	}
+	for d.NextBlock(0) {
+		order = append(order, d.Val())
+		if d.NextArg() {
+			return nil, d.ArgErr()
+		}
+	}
+	if len(order) == 0 {
+		return nil, d.ArgErr()
+	}
+	return order, nil
+}
+
+func parseOptStorage(d *caddyfile.Dispenser) (caddy.StorageConverter, error) {
+	if !d.Next() {
+		return nil, d.ArgErr()
+	}
+	args := d.RemainingArgs()
+	if len(args) != 1 {
+		return nil, d.ArgErr()
+	}
+	modName := args[0]
+	mod, err := caddy.GetModule("caddy.storage." + modName)
+	if err != nil {
+		return nil, fmt.Errorf("getting storage module '%s': %v", modName, err)
+	}
+	unm, ok := mod.New().(caddyfile.Unmarshaler)
+	if !ok {
+		return nil, fmt.Errorf("storage module '%s' is not a Caddyfile unmarshaler", mod.Name)
+	}
+	err = unm.UnmarshalCaddyfile(d.NewFromNextTokens())
+	if err != nil {
+		return nil, err
+	}
+	storage, ok := unm.(caddy.StorageConverter)
+	if !ok {
+		return nil, fmt.Errorf("module %s is not a StorageConverter", mod.Name)
+	}
+	return storage, nil
+}
+
+func parseOptACMECA(d *caddyfile.Dispenser) (string, error) {
+	d.Next() // consume parameter name
+	if !d.Next() {
+		return "", d.ArgErr()
+	}
+	val := d.Val()
+	if d.Next() {
+		return "", d.ArgErr()
+	}
+	return val, nil
+}
+
+func parseOptEmail(d *caddyfile.Dispenser) (string, error) {
+	d.Next() // consume parameter name
+	if !d.Next() {
+		return "", d.ArgErr()
+	}
+	val := d.Val()
+	if d.Next() {
+		return "", d.ArgErr()
+	}
+	return val, nil
+}

caddyconfig/json5/json5.go

@@ -0,0 +1,43 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package json5adapter
+
+import (
+	"encoding/json"
+
+	"github.com/caddyserver/caddy/v2/caddyconfig"
+	"github.com/ilibs/json5"
+)
+
+func init() {
+	caddyconfig.RegisterAdapter("json5", Adapter{})
+}
+
+// Adapter adapts JSON5 to Caddy JSON.
+type Adapter struct{}
+
+// Adapt converts the JSON5 config in body to Caddy JSON.
+func (a Adapter) Adapt(body []byte, options map[string]interface{}) (result []byte, warnings []caddyconfig.Warning, err error) {
+	var decoded interface{}
+	err = json5.Unmarshal(body, &decoded)
+	if err != nil {
+		return
+	}
+	result, err = json.Marshal(decoded)
+	return
+}
+
+// Interface guard
+var _ caddyconfig.Adapter = (*Adapter)(nil)

caddyconfig/jsonc/jsonc.go

@@ -0,0 +1,49 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package jsoncadapter
+
+import (
+	"encoding/json"
+
+	"github.com/caddyserver/caddy/v2/caddyconfig"
+	"github.com/muhammadmuzzammil1998/jsonc"
+)
+
+func init() {
+	caddyconfig.RegisterAdapter("jsonc", Adapter{})
+}
+
+// Adapter adapts JSON-C to Caddy JSON.
+type Adapter struct{}
+
+// Adapt converts the JSON-C config in body to Caddy JSON.
+func (a Adapter) Adapt(body []byte, options map[string]interface{}) (result []byte, warnings []caddyconfig.Warning, err error) {
+	result = jsonc.ToJSON(body)
+
+	// any errors in the JSON will be
+	// reported during config load, but
+	// we can at least warn here that
+	// it is not valid JSON
+	if !json.Valid(result) {
+		warnings = append(warnings, caddyconfig.Warning{
+			Message: "Resulting JSON is invalid.",
+		})
+	}
+
+	return
+}
+
+// Interface guard
+var _ caddyconfig.Adapter = (*Adapter)(nil)

cmd/caddy/main.go

@@ -0,0 +1,47 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package main
+
+import (
+	caddycmd "github.com/caddyserver/caddy/v2/cmd"
+
+	// this is where modules get plugged in
+	_ "github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
+	_ "github.com/caddyserver/caddy/v2/caddyconfig/json5"
+	_ "github.com/caddyserver/caddy/v2/caddyconfig/jsonc"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/caddyauth"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/encode"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/encode/brotli"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/encode/gzip"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/encode/zstd"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/fileserver"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/headers"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/httpcache"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/markdown"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/requestbody"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/reverseproxy/fastcgi"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/rewrite"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/starlarkmw"
+	_ "github.com/caddyserver/caddy/v2/modules/caddyhttp/templates"
+	_ "github.com/caddyserver/caddy/v2/modules/caddytls"
+	_ "github.com/caddyserver/caddy/v2/modules/caddytls/standardstek"
+	_ "github.com/caddyserver/caddy/v2/modules/filestorage"
+)
+
+func main() {
+	caddycmd.Main()
+}

cmd/commandfuncs.go

@@ -0,0 +1,523 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddycmd
+
+import (
+	"bytes"
+	"crypto/rand"
+	"encoding/json"
+	"fmt"
+	"io"
+	"io/ioutil"
+	"log"
+	"net"
+	"net/http"
+	"os"
+	"os/exec"
+	"reflect"
+	"runtime/debug"
+	"sort"
+	"strings"
+
+	"github.com/caddyserver/caddy/v2"
+	"github.com/caddyserver/caddy/v2/caddyconfig"
+	"github.com/keybase/go-ps"
+	"github.com/mholt/certmagic"
+)
+
+func cmdStart(fl Flags) (int, error) {
+	startCmdConfigFlag := fl.String("config")
+	startCmdConfigAdapterFlag := fl.String("adapter")
+
+	// open a listener to which the child process will connect when
+	// it is ready to confirm that it has successfully started
+	ln, err := net.Listen("tcp", "127.0.0.1:0")
+	if err != nil {
+		return caddy.ExitCodeFailedStartup,
+			fmt.Errorf("opening listener for success confirmation: %v", err)
+	}
+	defer ln.Close()
+
+	// craft the command with a pingback address and with a
+	// pipe for its stdin, so we can tell it our confirmation
+	// code that we expect so that some random port scan at
+	// the most unfortunate time won't fool us into thinking
+	// the child succeeded (i.e. the alternative is to just
+	// wait for any connection on our listener, but better to
+	// ensure it's the process we're expecting - we can be
+	// sure by giving it some random bytes and having it echo
+	// them back to us)
+	cmd := exec.Command(os.Args[0], "run", "--pingback", ln.Addr().String())
+	if startCmdConfigFlag != "" {
+		cmd.Args = append(cmd.Args, "--config", startCmdConfigFlag)
+	}
+	if startCmdConfigAdapterFlag != "" {
+		cmd.Args = append(cmd.Args, "--adapter", startCmdConfigAdapterFlag)
+	}
+	stdinpipe, err := cmd.StdinPipe()
+	if err != nil {
+		return caddy.ExitCodeFailedStartup,
+			fmt.Errorf("creating stdin pipe: %v", err)
+	}
+	cmd.Stdout = os.Stdout
+	cmd.Stderr = os.Stderr
+
+	// generate the random bytes we'll send to the child process
+	expect := make([]byte, 32)
+	_, err = rand.Read(expect)
+	if err != nil {
+		return caddy.ExitCodeFailedStartup, fmt.Errorf("generating random confirmation bytes: %v", err)
+	}
+
+	// begin writing the confirmation bytes to the child's
+	// stdin; use a goroutine since the child hasn't been
+	// started yet, and writing sychronously would result
+	// in a deadlock
+	go func() {
+		stdinpipe.Write(expect)
+		stdinpipe.Close()
+	}()
+
+	// start the process
+	err = cmd.Start()
+	if err != nil {
+		return caddy.ExitCodeFailedStartup, fmt.Errorf("starting caddy process: %v", err)
+	}
+
+	// there are two ways we know we're done: either
+	// the process will connect to our listener, or
+	// it will exit with an error
+	success, exit := make(chan struct{}), make(chan error)
+
+	// in one goroutine, we await the success of the child process
+	go func() {
+		for {
+			conn, err := ln.Accept()
+			if err != nil {
+				if !strings.Contains(err.Error(), "use of closed network connection") {
+					log.Println(err)
+				}
+				break
+			}
+			err = handlePingbackConn(conn, expect)
+			if err == nil {
+				close(success)
+				break
+			}
+			log.Println(err)
+		}
+	}()
+
+	// in another goroutine, we await the failure of the child process
+	go func() {
+		err := cmd.Wait() // don't send on this line! Wait blocks, but send starts before it unblocks
+		exit <- err       // sending on separate line ensures select won't trigger until after Wait unblocks
+	}()
+
+	// when one of the goroutines unblocks, we're done and can exit
+	select {
+	case <-success:
+		fmt.Printf("Successfully started Caddy (pid=%d)\n", cmd.Process.Pid)
+	case err := <-exit:
+		return caddy.ExitCodeFailedStartup,
+			fmt.Errorf("caddy process exited with error: %v", err)
+	}
+
+	return caddy.ExitCodeSuccess, nil
+}
+
+func cmdRun(fl Flags) (int, error) {
+	runCmdConfigFlag := fl.String("config")
+	runCmdConfigAdapterFlag := fl.String("adapter")
+	runCmdPrintEnvFlag := fl.Bool("environ")
+	runCmdPingbackFlag := fl.String("pingback")
+
+	// if we are supposed to print the environment, do that first
+	if runCmdPrintEnvFlag {
+		printEnvironment()
+	}
+
+	// get the config in caddy's native format
+	config, err := loadConfig(runCmdConfigFlag, runCmdConfigAdapterFlag)
+	if err != nil {
+		return caddy.ExitCodeFailedStartup, err
+	}
+
+	// set a fitting User-Agent for ACME requests
+	goModule := caddy.GoModule()
+	cleanModVersion := strings.TrimPrefix(goModule.Version, "v")
+	certmagic.UserAgent = "Caddy/" + cleanModVersion
+
+	// start the admin endpoint along with any initial config
+	err = caddy.StartAdmin(config)
+	if err != nil {
+		return caddy.ExitCodeFailedStartup,
+			fmt.Errorf("starting caddy administration endpoint: %v", err)
+	}
+	defer caddy.StopAdmin()
+
+	// if we are to report to another process the successful start
+	// of the server, do so now by echoing back contents of stdin
+	if runCmdPingbackFlag != "" {
+		confirmationBytes, err := ioutil.ReadAll(os.Stdin)
+		if err != nil {
+			return caddy.ExitCodeFailedStartup,
+				fmt.Errorf("reading confirmation bytes from stdin: %v", err)
+		}
+		conn, err := net.Dial("tcp", runCmdPingbackFlag)
+		if err != nil {
+			return caddy.ExitCodeFailedStartup,
+				fmt.Errorf("dialing confirmation address: %v", err)
+		}
+		defer conn.Close()
+		_, err = conn.Write(confirmationBytes)
+		if err != nil {
+			return caddy.ExitCodeFailedStartup,
+				fmt.Errorf("writing confirmation bytes to %s: %v", runCmdPingbackFlag, err)
+		}
+	}
+
+	select {}
+}
+
+func cmdStop(_ Flags) (int, error) {
+	processList, err := ps.Processes()
+	if err != nil {
+		return caddy.ExitCodeFailedStartup, fmt.Errorf("listing processes: %v", err)
+	}
+	thisProcName := getProcessName()
+	var found bool
+	for _, p := range processList {
+		// the process we're looking for should have the same name but different PID
+		if p.Executable() == thisProcName && p.Pid() != os.Getpid() {
+			found = true
+			fmt.Printf("pid=%d\n", p.Pid())
+
+			if err := gracefullyStopProcess(p.Pid()); err != nil {
+				return caddy.ExitCodeFailedStartup, err
+			}
+		}
+	}
+	if !found {
+		return caddy.ExitCodeFailedStartup, fmt.Errorf("Caddy is not running")
+	}
+	fmt.Println(" success")
+	return caddy.ExitCodeSuccess, nil
+}
+
+func cmdReload(fl Flags) (int, error) {
+	reloadCmdConfigFlag := fl.String("config")
+	reloadCmdConfigAdapterFlag := fl.String("adapter")
+	reloadCmdAddrFlag := fl.String("address")
+
+	// a configuration is required
+	if reloadCmdConfigFlag == "" {
+		return caddy.ExitCodeFailedStartup,
+			fmt.Errorf("no configuration to load (use --config)")
+	}
+
+	// get the config in caddy's native format
+	config, err := loadConfig(reloadCmdConfigFlag, reloadCmdConfigAdapterFlag)
+	if err != nil {
+		return caddy.ExitCodeFailedStartup, err
+	}
+
+	// get the address of the admin listener and craft endpoint URL
+	adminAddr := reloadCmdAddrFlag
+	if adminAddr == "" {
+		var tmpStruct struct {
+			Admin caddy.AdminConfig `json:"admin"`
+		}
+		err = json.Unmarshal(config, &tmpStruct)
+		if err != nil {
+			return caddy.ExitCodeFailedStartup,
+				fmt.Errorf("unmarshaling admin listener address from config: %v", err)
+		}
+		adminAddr = tmpStruct.Admin.Listen
+	}
+	if adminAddr == "" {
+		adminAddr = caddy.DefaultAdminListen
+	}
+	adminEndpoint := fmt.Sprintf("http://%s/load", adminAddr)
+
+	// send the configuration to the instance
+	resp, err := http.Post(adminEndpoint, "application/json", bytes.NewReader(config))
+	if err != nil {
+		return caddy.ExitCodeFailedStartup,
+			fmt.Errorf("sending configuration to instance: %v", err)
+	}
+	defer resp.Body.Close()
+
+	// if it didn't work, let the user know
+	if resp.StatusCode >= 400 {
+		respBody, err := ioutil.ReadAll(io.LimitReader(resp.Body, 1024*10))
+		if err != nil {
+			return caddy.ExitCodeFailedStartup,
+				fmt.Errorf("HTTP %d: reading error message: %v", resp.StatusCode, err)
+		}
+		return caddy.ExitCodeFailedStartup,
+			fmt.Errorf("caddy responded with error: HTTP %d: %s", resp.StatusCode, respBody)
+	}
+
+	return caddy.ExitCodeSuccess, nil
+}
+
+func cmdVersion(_ Flags) (int, error) {
+	goModule := caddy.GoModule()
+	if goModule.Sum != "" {
+		// a build with a known version will also have a checksum
+		fmt.Printf("%s %s\n", goModule.Version, goModule.Sum)
+	} else {
+		fmt.Println(goModule.Version)
+	}
+	return caddy.ExitCodeSuccess, nil
+}
+
+func cmdListModules(fl Flags) (int, error) {
+	versions := fl.Bool("versions")
+
+	bi, ok := debug.ReadBuildInfo()
+	if !ok || !versions {
+		// if there's no build information,
+		// just print out the modules
+		for _, m := range caddy.Modules() {
+			fmt.Println(m)
+		}
+		return caddy.ExitCodeSuccess, nil
+	}
+
+	for _, modName := range caddy.Modules() {
+		modInfo, err := caddy.GetModule(modName)
+		if err != nil {
+			// that's weird
+			fmt.Println(modName)
+			continue
+		}
+
+		// to get the Caddy plugin's version info, we need to know
+		// the package that the Caddy module's value comes from; we
+		// can use reflection but we need a non-pointer value (I'm
+		// not sure why), and since New() should return a pointer
+		// value, we need to dereference it first
+		iface := interface{}(modInfo.New())
+		if rv := reflect.ValueOf(iface); rv.Kind() == reflect.Ptr {
+			iface = reflect.New(reflect.TypeOf(iface).Elem()).Elem().Interface()
+		}
+		modPkgPath := reflect.TypeOf(iface).PkgPath()
+
+		// now we find the Go module that the Caddy module's package
+		// belongs to; we assume the Caddy module package path will
+		// be prefixed by its Go module path, and we will choose the
+		// longest matching prefix in case there are nested modules
+		var matched *debug.Module
+		for _, dep := range bi.Deps {
+			if strings.HasPrefix(modPkgPath, dep.Path) {
+				if matched == nil || len(dep.Path) > len(matched.Path) {
+					matched = dep
+				}
+			}
+		}
+
+		// if we could find no matching module, just print out
+		// the module name instead
+		if matched == nil {
+			fmt.Println(modName)
+			continue
+		}
+
+		fmt.Printf("%s %s\n", modName, matched.Version)
+	}
+
+	return caddy.ExitCodeSuccess, nil
+}
+
+func cmdEnviron(_ Flags) (int, error) {
+	printEnvironment()
+	return caddy.ExitCodeSuccess, nil
+}
+
+func cmdAdaptConfig(fl Flags) (int, error) {
+	adaptCmdInputFlag := fl.String("config")
+	adaptCmdAdapterFlag := fl.String("adapter")
+	adaptCmdPrettyFlag := fl.Bool("pretty")
+	adaptCmdValidateFlag := fl.Bool("validate")
+
+	if adaptCmdAdapterFlag == "" || adaptCmdInputFlag == "" {
+		return caddy.ExitCodeFailedStartup,
+			fmt.Errorf("--adapter and --config flags are required")
+	}
+
+	cfgAdapter := caddyconfig.GetAdapter(adaptCmdAdapterFlag)
+	if cfgAdapter == nil {
+		return caddy.ExitCodeFailedStartup,
+			fmt.Errorf("unrecognized config adapter: %s", adaptCmdAdapterFlag)
+	}
+
+	input, err := ioutil.ReadFile(adaptCmdInputFlag)
+	if err != nil {
+		return caddy.ExitCodeFailedStartup,
+			fmt.Errorf("reading input file: %v", err)
+	}
+
+	opts := make(map[string]interface{})
+	if adaptCmdPrettyFlag {
+		opts["pretty"] = "true"
+	}
+
+	adaptedConfig, warnings, err := cfgAdapter.Adapt(input, opts)
+	if err != nil {
+		return caddy.ExitCodeFailedStartup, err
+	}
+
+	// print warnings to stderr
+	for _, warn := range warnings {
+		msg := warn.Message
+		if warn.Directive != "" {
+			msg = fmt.Sprintf("%s: %s", warn.Directive, warn.Message)
+		}
+		log.Printf("[WARNING][%s] %s:%d: %s", adaptCmdAdapterFlag, warn.File, warn.Line, msg)
+	}
+
+	// print result to stdout
+	fmt.Println(string(adaptedConfig))
+
+	// validate output if requested
+	if adaptCmdValidateFlag {
+		var cfg *caddy.Config
+		err = json.Unmarshal(adaptedConfig, &cfg)
+		if err != nil {
+			return caddy.ExitCodeFailedStartup, fmt.Errorf("decoding config: %v", err)
+		}
+		err = caddy.Validate(cfg)
+		if err != nil {
+			return caddy.ExitCodeFailedStartup, fmt.Errorf("validation: %v", err)
+		}
+	}
+
+	return caddy.ExitCodeSuccess, nil
+}
+
+func cmdValidateConfig(fl Flags) (int, error) {
+	validateCmdConfigFlag := fl.String("config")
+	validateCmdAdapterFlag := fl.String("adapter")
+
+	input, err := ioutil.ReadFile(validateCmdConfigFlag)
+	if err != nil {
+		return caddy.ExitCodeFailedStartup,
+			fmt.Errorf("reading input file: %v", err)
+	}
+
+	if validateCmdAdapterFlag != "" {
+		cfgAdapter := caddyconfig.GetAdapter(validateCmdAdapterFlag)
+		if cfgAdapter == nil {
+			return caddy.ExitCodeFailedStartup,
+				fmt.Errorf("unrecognized config adapter: %s", validateCmdAdapterFlag)
+		}
+
+		adaptedConfig, warnings, err := cfgAdapter.Adapt(input, nil)
+		if err != nil {
+			return caddy.ExitCodeFailedStartup, err
+		}
+		// print warnings to stderr
+		for _, warn := range warnings {
+			msg := warn.Message
+			if warn.Directive != "" {
+				msg = fmt.Sprintf("%s: %s", warn.Directive, warn.Message)
+			}
+			log.Printf("[WARNING][%s] %s:%d: %s", validateCmdAdapterFlag, warn.File, warn.Line, msg)
+		}
+
+		input = adaptedConfig
+	}
+
+	var cfg *caddy.Config
+	err = json.Unmarshal(input, &cfg)
+	if err != nil {
+		return caddy.ExitCodeFailedStartup, fmt.Errorf("decoding config: %v", err)
+	}
+
+	err = caddy.Validate(cfg)
+	if err != nil {
+		return caddy.ExitCodeFailedStartup, err
+	}
+
+	fmt.Println("Valid configuration")
+
+	return caddy.ExitCodeSuccess, nil
+}
+
+func cmdHelp(fl Flags) (int, error) {
+	const fullDocs = `Full documentation is available at:
+https://github.com/caddyserver/caddy/wiki/v2:-Documentation`
+
+	args := fl.Args()
+	if len(args) == 0 {
+		s := `Caddy is an extensible server platform.
+
+usage:
+  caddy <command> [<args...>]
+
+commands:
+`
+		keys := make([]string, 0, len(commands))
+		for k := range commands {
+			keys = append(keys, k)
+		}
+		sort.Strings(keys)
+		for _, k := range keys {
+			cmd := commands[k]
+			short := strings.TrimSuffix(cmd.Short, ".")
+			s += fmt.Sprintf("  %-15s %s\n", cmd.Name, short)
+		}
+
+		s += "\nUse 'caddy help <command>' for more information about a command.\n"
+		s += "\n" + fullDocs + "\n"
+
+		fmt.Print(s)
+
+		return caddy.ExitCodeSuccess, nil
+	} else if len(args) > 1 {
+		return caddy.ExitCodeFailedStartup, fmt.Errorf("can only give help with one command")
+	}
+
+	subcommand, ok := commands[args[0]]
+	if !ok {
+		return caddy.ExitCodeFailedStartup, fmt.Errorf("unknown command: %s", args[0])
+	}
+
+	helpText := strings.TrimSpace(subcommand.Long)
+	if helpText == "" {
+		helpText = subcommand.Short
+		if !strings.HasSuffix(helpText, ".") {
+			helpText += "."
+		}
+	}
+
+	result := fmt.Sprintf("%s\n\nusage:\n  caddy %s %s\n",
+		helpText,
+		subcommand.Name,
+		strings.TrimSpace(subcommand.Usage),
+	)
+
+	if help := flagHelp(subcommand.Flags); help != "" {
+		result += fmt.Sprintf("\nflags:\n%s", help)
+	}
+
+	result += "\n" + fullDocs + "\n"
+
+	fmt.Print(result)
+
+	return caddy.ExitCodeSuccess, nil
+}

cmd/commands.go

@@ -0,0 +1,265 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddycmd
+
+import (
+	"flag"
+	"regexp"
+)
+
+// Command represents a subcommand. Name, Func,
+// and Short are required.
+type Command struct {
+	// The name of the subcommand. Must conform to the
+	// format described by the RegisterCommand() godoc.
+	// Required.
+	Name string
+
+	// Run is a function that executes a subcommand using
+	// the parsed flags. It returns an exit code and any
+	// associated error.
+	// Required.
+	Func CommandFunc
+
+	// Usage is a brief message describing the syntax of
+	// the subcommand's flags and args. Use [] to indicate
+	// optional parameters and <> to enclose literal values
+	// intended to be replaced by the user. Do not prefix
+	// the string with "caddy" or the name of the command
+	// since these will be prepended for you; only include
+	// the actual parameters for this command.
+	Usage string
+
+	// Short is a one-line message explaining what the
+	// command does. Should not end with punctuation.
+	// Required.
+	Short string
+
+	// Long is the full help text shown to the user.
+	// Will be trimmed of whitespace on both ends before
+	// being printed.
+	Long string
+
+	// Flags is the flagset for command.
+	Flags *flag.FlagSet
+}
+
+// CommandFunc is a command's function. It runs the
+// command and returns the proper exit code along with
+// any error that occurred.
+type CommandFunc func(Flags) (int, error)
+
+var commands = make(map[string]Command)
+
+func init() {
+	RegisterCommand(Command{
+		Name:  "help",
+		Func:  cmdHelp,
+		Usage: "<command>",
+		Short: "Shows help for a Caddy subcommand",
+	})
+
+	RegisterCommand(Command{
+		Name:  "start",
+		Func:  cmdStart,
+		Usage: "[--config <path> [[--adapter <name>]]",
+		Short: "Starts the Caddy process in the background and then returns",
+		Long: `
+Starts the Caddy process, optionally bootstrapped with an initial config file.
+This command unblocks after the server starts running or fails to run.
+
+On Windows, the spawned child process will remain attached to the terminal, so
+closing the window will forcefully stop Caddy; to avoid forgetting this, try
+using 'caddy run' instead to keep it in the foreground.`,
+		Flags: func() *flag.FlagSet {
+			fs := flag.NewFlagSet("start", flag.ExitOnError)
+			fs.String("config", "", "Configuration file")
+			fs.String("adapter", "", "Name of config adapter to apply")
+			return fs
+		}(),
+	})
+
+	RegisterCommand(Command{
+		Name:  "run",
+		Func:  cmdRun,
+		Usage: "[--config <path> [--adapter <name>]] [--environ]",
+		Short: `Starts the Caddy process and blocks indefinitely`,
+		Long: `
+Starts the Caddy process, optionally bootstrapped with an initial config file,
+and blocks indefinitely until the server is stopped; i.e. runs Caddy in
+"daemon" mode (foreground).
+
+If a config file is specified, it will be applied immediately after the process
+is running. If the config file is not in Caddy's native JSON format, you can
+specify an adapter with --adapter to adapt the given config file to
+Caddy's native format. The config adapter must be a registered module. Any
+warnings will be printed to the log, but beware that any adaptation without
+errors will immediately be used. If you want to review the results of the
+adaptation first, use the 'adapt' subcommand.
+
+As a special case, if the current working directory has a file called
+"Caddyfile" and the caddyfile config adapter is plugged in (default), then
+that file will be loaded and used to configure Caddy, even without any command
+line flags.
+
+If --environ is specified, the environment as seen by the Caddy process will
+be printed before starting. This is the same as the environ command but does
+not quit after printing, and can be useful for troubleshooting.`,
+		Flags: func() *flag.FlagSet {
+			fs := flag.NewFlagSet("run", flag.ExitOnError)
+			fs.String("config", "", "Configuration file")
+			fs.String("adapter", "", "Name of config adapter to apply")
+			fs.Bool("environ", false, "Print environment")
+			fs.String("pingback", "", "Echo confirmation bytes to this address on success")
+			return fs
+		}(),
+	})
+
+	RegisterCommand(Command{
+		Name:  "stop",
+		Func:  cmdStop,
+		Short: "Gracefully stops the running Caddy process",
+		Long: `
+Stops the running Caddy process as gracefully as possible.
+
+On Windows, this stop is forceful and Caddy will not have an opportunity to
+clean up any active locks; for a graceful shutdown on Windows, use Ctrl+C
+or the /stop API endpoint.
+
+Note: this will stop any process named the same as the executable (os.Args[0]).`,
+	})
+
+	RegisterCommand(Command{
+		Name:  "reload",
+		Func:  cmdReload,
+		Usage: "--config <path> [--adapter <name>] [--address <interface>]",
+		Short: "Changes the config of the running Caddy instance",
+		Long: `
+Gives the running Caddy instance a new configuration. This has the same effect
+as POSTing a document to the /load API endpoint, but is convenient for simple
+workflows revolving around config files.
+
+Since the admin endpoint is configurable, the endpoint configuration is loaded
+from the --address flag if specified; otherwise it is loaded from the given
+config file; otherwise the default is assumed.`,
+		Flags: func() *flag.FlagSet {
+			fs := flag.NewFlagSet("reload", flag.ExitOnError)
+			fs.String("config", "", "Configuration file (required)")
+			fs.String("adapter", "", "Name of config adapter to apply")
+			fs.String("address", "", "Address of the administration listener, if different from config")
+			return fs
+		}(),
+	})
+
+	RegisterCommand(Command{
+		Name:  "version",
+		Func:  cmdVersion,
+		Short: "Prints the version",
+	})
+
+	RegisterCommand(Command{
+		Name:  "list-modules",
+		Func:  cmdListModules,
+		Usage: "[--versions]",
+		Short: "Lists the installed Caddy modules",
+		Flags: func() *flag.FlagSet {
+			fs := flag.NewFlagSet("list-modules", flag.ExitOnError)
+			fs.Bool("versions", false, "Print version information")
+			return fs
+		}(),
+	})
+
+	RegisterCommand(Command{
+		Name:  "environ",
+		Func:  cmdEnviron,
+		Short: "Prints the environment",
+	})
+
+	RegisterCommand(Command{
+		Name:  "adapt",
+		Func:  cmdAdaptConfig,
+		Usage: "--config <path> [--adapter <name>] [--pretty] [--validate]",
+		Short: "Adapts a configuration to Caddy's native JSON",
+		Long: `
+Adapts a configuration to Caddy's native JSON format and writes the
+output to stdout, along with any warnings to stderr.
+
+If --pretty is specified, the output will be formatted with indentation
+for human readability.
+
+If --validate is used, the adapted config will be checked for validity.
+If the config is invalid, an error will be printed to stderr and a non-
+zero exit status will be returned.`,
+		Flags: func() *flag.FlagSet {
+			fs := flag.NewFlagSet("adapt", flag.ExitOnError)
+			fs.String("config", "", "Configuration file to adapt (required)")
+			fs.String("adapter", "caddyfile", "Name of config adapter")
+			fs.Bool("pretty", false, "Format the output for human readability")
+			fs.Bool("validate", false, "Validate the output")
+			return fs
+		}(),
+	})
+
+	RegisterCommand(Command{
+		Name:  "validate",
+		Func:  cmdValidateConfig,
+		Usage: "--config <path> [--adapter <name>]",
+		Short: "Tests whether a configuration file is valid",
+		Long: `
+Loads and provisions the provided config, but does not start running it.
+This reveals any errors with the configuration through the loading and
+provisioning stages.`,
+		Flags: func() *flag.FlagSet {
+			fs := flag.NewFlagSet("load", flag.ExitOnError)
+			fs.String("config", "", "Input configuration file")
+			fs.String("adapter", "", "Name of config adapter")
+			return fs
+		}(),
+	})
+
+}
+
+// RegisterCommand registers the command cmd.
+// cmd.Name must be unique and conform to the
+// following format:
+//
+//    - lowercase
+//    - alphanumeric and hyphen characters only
+//    - cannot start or end with a hyphen
+//    - hyphen cannot be adjacent to another hyphen
+//
+// This function panics if the name is already registered,
+// if the name does not meet the described format, or if
+// any of the fields are missing from cmd.
+func RegisterCommand(cmd Command) {
+	if cmd.Name == "" {
+		panic("command name is required")
+	}
+	if cmd.Func == nil {
+		panic("command function missing")
+	}
+	if cmd.Short == "" {
+		panic("command short string is required")
+	}
+	if _, exists := commands[cmd.Name]; exists {
+		panic("command already registered: " + cmd.Name)
+	}
+	if !commandNameRegex.MatchString(cmd.Name) {
+		panic("invalid command name")
+	}
+	commands[cmd.Name] = cmd
+}
+
+var commandNameRegex = regexp.MustCompile(`^[a-z0-9]$|^([a-z0-9]+-?[a-z0-9]*)+[a-z0-9]$`)

cmd/main.go

@@ -0,0 +1,230 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddycmd
+
+import (
+	"bytes"
+	"flag"
+	"fmt"
+	"io"
+	"io/ioutil"
+	"log"
+	"net"
+	"os"
+	"strconv"
+	"strings"
+	"time"
+
+	"github.com/caddyserver/caddy/v2"
+	"github.com/caddyserver/caddy/v2/caddyconfig"
+)
+
+// Main implements the main function of the caddy command.
+// Call this if Caddy is to be the main() if your program.
+func Main() {
+	caddy.TrapSignals()
+
+	switch len(os.Args) {
+	case 0:
+		log.Printf("[FATAL] no arguments provided by OS; args[0] must be command")
+		os.Exit(caddy.ExitCodeFailedStartup)
+	case 1:
+		os.Args = append(os.Args, "help")
+	}
+
+	subcommandName := os.Args[1]
+	subcommand, ok := commands[subcommandName]
+	if !ok {
+		if strings.HasPrefix(os.Args[1], "-") {
+			// user probably forgot to type the subcommand
+			log.Println("[ERROR] first argument must be a subcommand; see 'caddy help'")
+		} else {
+			log.Printf("[ERROR] '%s' is not a recognized subcommand; see 'caddy help'", os.Args[1])
+		}
+		os.Exit(caddy.ExitCodeFailedStartup)
+	}
+
+	fs := subcommand.Flags
+	if fs == nil {
+		fs = flag.NewFlagSet(subcommand.Name, flag.ExitOnError)
+	}
+
+	err := fs.Parse(os.Args[2:])
+	if err != nil {
+		log.Println(err)
+		os.Exit(caddy.ExitCodeFailedStartup)
+	}
+
+	exitCode, err := subcommand.Func(Flags{fs})
+	if err != nil {
+		log.Printf("%s: %v", subcommand.Name, err)
+	}
+
+	os.Exit(exitCode)
+}
+
+// handlePingbackConn reads from conn and ensures it matches
+// the bytes in expect, or returns an error if it doesn't.
+func handlePingbackConn(conn net.Conn, expect []byte) error {
+	defer conn.Close()
+	confirmationBytes, err := ioutil.ReadAll(io.LimitReader(conn, 32))
+	if err != nil {
+		return err
+	}
+	if !bytes.Equal(confirmationBytes, expect) {
+		return fmt.Errorf("wrong confirmation: %x", confirmationBytes)
+	}
+	return nil
+}
+
+// loadConfig loads the config from configFile and adapts it
+// using adapterName. If adapterName is specified, configFile
+// must be also. It prints any warnings to stderr, and returns
+// the resulting JSON config bytes.
+func loadConfig(configFile, adapterName string) ([]byte, error) {
+	// specifying an adapter without a config file is ambiguous
+	if configFile == "" && adapterName != "" {
+		return nil, fmt.Errorf("cannot adapt config without config file (use --config)")
+	}
+
+	// load initial config and adapter
+	var config []byte
+	var cfgAdapter caddyconfig.Adapter
+	var err error
+	if configFile != "" {
+		config, err = ioutil.ReadFile(configFile)
+		if err != nil {
+			return nil, fmt.Errorf("reading config file: %v", err)
+		}
+	} else if adapterName == "" {
+		// as a special case when no config file or adapter
+		// is specified, see if the Caddyfile adapter is
+		// plugged in, and if so, try using a default Caddyfile
+		cfgAdapter = caddyconfig.GetAdapter("caddyfile")
+		if cfgAdapter != nil {
+			config, err = ioutil.ReadFile("Caddyfile")
+			if os.IsNotExist(err) {
+				// okay, no default Caddyfile; pretend like this never happened
+				cfgAdapter = nil
+				err = nil
+			} else if err != nil {
+				// default Caddyfile exists, but error reading it
+				return nil, fmt.Errorf("reading default Caddyfile: %v", err)
+			} else {
+				// success reading default Caddyfile
+				configFile = "Caddyfile"
+			}
+		}
+	}
+
+	// load config adapter
+	if adapterName != "" {
+		cfgAdapter = caddyconfig.GetAdapter(adapterName)
+		if cfgAdapter == nil {
+			return nil, fmt.Errorf("unrecognized config adapter: %s", adapterName)
+		}
+	}
+
+	// adapt config
+	if cfgAdapter != nil {
+		adaptedConfig, warnings, err := cfgAdapter.Adapt(config, map[string]interface{}{
+			"filename": configFile,
+		})
+		if err != nil {
+			return nil, fmt.Errorf("adapting config using %s: %v", adapterName, err)
+		}
+		for _, warn := range warnings {
+			msg := warn.Message
+			if warn.Directive != "" {
+				msg = fmt.Sprintf("%s: %s", warn.Directive, warn.Message)
+			}
+			fmt.Printf("[WARNING][%s] %s:%d: %s", adapterName, warn.File, warn.Line, msg)
+		}
+		config = adaptedConfig
+	}
+
+	return config, nil
+}
+
+// Flags wraps a FlagSet so that typed values
+// from flags can be easily retrieved.
+type Flags struct {
+	*flag.FlagSet
+}
+
+// String returns the string representation of the
+// flag given by name. It panics if the flag is not
+// in the flag set.
+func (f Flags) String(name string) string {
+	return f.FlagSet.Lookup(name).Value.String()
+}
+
+// Bool returns the boolean representation of the
+// flag given by name. It returns false if the flag
+// is not a boolean type. It panics if the flag is
+// not in the flag set.
+func (f Flags) Bool(name string) bool {
+	val, _ := strconv.ParseBool(f.String(name))
+	return val
+}
+
+// Int returns the integer representation of the
+// flag given by name. It returns 0 if the flag
+// is not an integer type. It panics if the flag is
+// not in the flag set.
+func (f Flags) Int(name string) int {
+	val, _ := strconv.ParseInt(f.String(name), 0, strconv.IntSize)
+	return int(val)
+}
+
+// Float64 returns the float64 representation of the
+// flag given by name. It returns false if the flag
+// is not a float63 type. It panics if the flag is
+// not in the flag set.
+func (f Flags) Float64(name string) float64 {
+	val, _ := strconv.ParseFloat(f.String(name), 64)
+	return val
+}
+
+// Duration returns the duration representation of the
+// flag given by name. It returns false if the flag
+// is not a duration type. It panics if the flag is
+// not in the flag set.
+func (f Flags) Duration(name string) time.Duration {
+	val, _ := time.ParseDuration(f.String(name))
+	return val
+}
+
+// flagHelp returns the help text for fs.
+func flagHelp(fs *flag.FlagSet) string {
+	if fs == nil {
+		return ""
+	}
+
+	// temporarily redirect output
+	out := fs.Output()
+	defer fs.SetOutput(out)
+
+	buf := new(bytes.Buffer)
+	fs.SetOutput(buf)
+	fs.PrintDefaults()
+	return buf.String()
+}
+
+func printEnvironment() {
+	for _, v := range os.Environ() {
+		fmt.Println(v)
+	}
+}

cmd/proc_posix.go

@@ -0,0 +1,37 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// +build !windows
+
+package caddycmd
+
+import (
+	"fmt"
+	"os"
+	"path/filepath"
+	"syscall"
+)
+
+func gracefullyStopProcess(pid int) error {
+	fmt.Printf("Graceful stop...")
+	err := syscall.Kill(pid, syscall.SIGINT)
+	if err != nil {
+		return fmt.Errorf("kill: %v", err)
+	}
+	return nil
+}
+
+func getProcessName() string {
+	return filepath.Base(os.Args[0])
+}

cmd/proc_windows.go

@@ -0,0 +1,44 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddycmd
+
+import (
+	"fmt"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strconv"
+)
+
+func gracefullyStopProcess(pid int) error {
+	fmt.Printf("Forceful Stop...")
+	// process on windows will not stop unless forced with /f
+	cmd := exec.Command("taskkill", "/pid", strconv.Itoa(pid), "/f")
+	if err := cmd.Run(); err != nil {
+		return fmt.Errorf("taskkill: %v", err)
+	}
+	return nil
+}
+
+// On Windows the app name passed in os.Args[0] will match how
+// caddy was started eg will match caddy or caddy.exe.
+// So return appname with .exe for consistency
+func getProcessName() string {
+	base := filepath.Base(os.Args[0])
+	if filepath.Ext(base) == "" {
+		return base + ".exe"
+	}
+	return base
+}

config/config.go

@@ -1,169 +0,0 @@
-package config
-
-import (
-	"errors"
-	"fmt"
-	"io"
-	"log"
-	"net"
-
-	"github.com/mholt/caddy/app"
-	"github.com/mholt/caddy/config/parse"
-	"github.com/mholt/caddy/config/setup"
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/server"
-)
-
-const (
-	DefaultHost = "0.0.0.0"
-	DefaultPort = "2015"
-	DefaultRoot = "."
-
-	// DefaultConfigFile is the name of the configuration file that is loaded
-	// by default if no other file is specified.
-	DefaultConfigFile = "Caddyfile"
-)
-
-func Load(filename string, input io.Reader) ([]server.Config, error) {
-	var configs []server.Config
-
-	// turn off timestamp for parsing
-	flags := log.Flags()
-	log.SetFlags(0)
-
-	serverBlocks, err := parse.ServerBlocks(filename, input)
-	if err != nil {
-		return configs, err
-	}
-
-	// Each server block represents a single server/address.
-	// Iterate each server block and make a config for each one,
-	// executing the directives that were parsed.
-	for _, sb := range serverBlocks {
-		config := server.Config{
-			Host:       sb.Host,
-			Port:       sb.Port,
-			Root:       Root,
-			Middleware: make(map[string][]middleware.Middleware),
-			ConfigFile: filename,
-			AppName:    app.Name,
-			AppVersion: app.Version,
-		}
-
-		// It is crucial that directives are executed in the proper order.
-		for _, dir := range directiveOrder {
-			// Execute directive if it is in the server block
-			if tokens, ok := sb.Tokens[dir.name]; ok {
-				// Each setup function gets a controller, which is the
-				// server config and the dispenser containing only
-				// this directive's tokens.
-				controller := &setup.Controller{
-					Config:    &config,
-					Dispenser: parse.NewDispenserTokens(filename, tokens),
-				}
-
-				midware, err := dir.setup(controller)
-				if err != nil {
-					return configs, err
-				}
-				if midware != nil {
-					// TODO: For now, we only support the default path scope /
-					config.Middleware["/"] = append(config.Middleware["/"], midware)
-				}
-			}
-		}
-
-		if config.Port == "" {
-			config.Port = Port
-		}
-
-		configs = append(configs, config)
-	}
-
-	// restore logging settings
-	log.SetFlags(flags)
-
-	return configs, nil
-}
-
-// ArrangeBindings groups configurations by their bind address. For example,
-// a server that should listen on localhost and another on 127.0.0.1 will
-// be grouped into the same address: 127.0.0.1. It will return an error
-// if the address lookup fails or if a TLS listener is configured on the
-// same address as a plaintext HTTP listener. The return value is a map of
-// bind address to list of configs that would become VirtualHosts on that
-// server.
-func ArrangeBindings(allConfigs []server.Config) (map[*net.TCPAddr][]server.Config, error) {
-	addresses := make(map[*net.TCPAddr][]server.Config)
-
-	// Group configs by bind address
-	for _, conf := range allConfigs {
-		newAddr, err := net.ResolveTCPAddr("tcp", conf.Address())
-		if err != nil {
-			return addresses, errors.New("Could not serve " + conf.Address() + " - " + err.Error())
-		}
-
-		// Make sure to compare the string representation of the address,
-		// not the pointer, since a new *TCPAddr is created each time.
-		var existing bool
-		for addr := range addresses {
-			if addr.String() == newAddr.String() {
-				addresses[addr] = append(addresses[addr], conf)
-				existing = true
-				break
-			}
-		}
-		if !existing {
-			addresses[newAddr] = append(addresses[newAddr], conf)
-		}
-	}
-
-	// Don't allow HTTP and HTTPS to be served on the same address
-	for _, configs := range addresses {
-		isTLS := configs[0].TLS.Enabled
-		for _, config := range configs {
-			if config.TLS.Enabled != isTLS {
-				thisConfigProto, otherConfigProto := "HTTP", "HTTP"
-				if config.TLS.Enabled {
-					thisConfigProto = "HTTPS"
-				}
-				if configs[0].TLS.Enabled {
-					otherConfigProto = "HTTPS"
-				}
-				return addresses, fmt.Errorf("Configuration error: Cannot multiplex %s (%s) and %s (%s) on same address",
-					configs[0].Address(), otherConfigProto, config.Address(), thisConfigProto)
-			}
-		}
-	}
-
-	return addresses, nil
-}
-
-// validDirective returns true if d is a valid
-// directive; false otherwise.
-func validDirective(d string) bool {
-	for _, dir := range directiveOrder {
-		if dir.name == d {
-			return true
-		}
-	}
-	return false
-}
-
-// Default makes a default configuration which
-// is empty except for root, host, and port,
-// which are essentials for serving the cwd.
-func Default() server.Config {
-	return server.Config{
-		Root: Root,
-		Host: Host,
-		Port: Port,
-	}
-}
-
-// These three defaults are configurable through the command line
-var (
-	Root = DefaultRoot
-	Host = DefaultHost
-	Port = DefaultPort
-)

config/directives.go

@@ -1,80 +0,0 @@
-package config
-
-import (
-	"github.com/mholt/caddy/config/parse"
-	"github.com/mholt/caddy/config/setup"
-	"github.com/mholt/caddy/middleware"
-)
-
-func init() {
-	// The parse package must know which directives
-	// are valid, but it must not import the setup
-	// or config package. To solve this problem, we
-	// fill up this map in our init function here.
-	// The parse package does not need to know the
-	// ordering of the directives.
-	for _, dir := range directiveOrder {
-		parse.ValidDirectives[dir.name] = struct{}{}
-	}
-}
-
-// Directives are registered in the order they should be
-// executed. Middleware (directives that inject a handler)
-// are executed in the order A-B-C-*-C-B-A, assuming
-// they all call the Next handler in the chain.
-//
-// Ordering is VERY important. Every middleware will
-// feel the effects of all other middleware below
-// (after) them during a request, but they must not
-// care what middleware above them are doing.
-//
-// For example, log needs to know the status code and
-// exactly how many bytes were written to the client,
-// which every other middleware can affect, so it gets
-// registered first. The errors middleware does not
-// care if gzip or log modifies its response, so it
-// gets registered below them. Gzip, on the other hand,
-// DOES care what errors does to the response since it
-// must compress every output to the client, even error
-// pages, so it must be registered before the errors
-// middleware and any others that would write to the
-// response.
-var directiveOrder = []directive{
-	// Essential directives that initialize vital configuration settings
-	{"root", setup.Root},
-	{"tls", setup.TLS},
-	{"bind", setup.BindHost},
-
-	// Other directives that don't create HTTP handlers
-	{"startup", setup.Startup},
-	{"shutdown", setup.Shutdown},
-	{"git", setup.Git},
-
-	// Directives that inject handlers (middleware)
-	{"log", setup.Log},
-	{"gzip", setup.Gzip},
-	{"errors", setup.Errors},
-	{"header", setup.Headers},
-	{"rewrite", setup.Rewrite},
-	{"redir", setup.Redir},
-	{"ext", setup.Ext},
-	{"basicauth", setup.BasicAuth},
-	{"internal", setup.Internal},
-	{"proxy", setup.Proxy},
-	{"fastcgi", setup.FastCGI},
-	{"websocket", setup.WebSocket},
-	{"markdown", setup.Markdown},
-	{"templates", setup.Templates},
-	{"browse", setup.Browse},
-}
-
-// directive ties together a directive name with its setup function.
-type directive struct {
-	name  string
-	setup setupFunc
-}
-
-// A setup function takes a setup controller. Its return values may
-// both be nil. If middleware is not nil, it will be chained into
-// the HTTP handlers in the order specified in this package.
-type setupFunc func(c *setup.Controller) (middleware.Middleware, error)

config/parse/dispenser.go

@@ -1,217 +0,0 @@
-package parse
-
-import (
-	"errors"
-	"fmt"
-	"io"
-	"strings"
-)
-
-// Dispenser is a type that dispenses tokens, similarly to a lexer,
-// except that it can do so with some notion of structure and has
-// some really convenient methods.
-type Dispenser struct {
-	filename string
-	tokens   []token
-	cursor   int
-	nesting  int
-}
-
-// NewDispenser returns a Dispenser, ready to use for parsing the given input.
-func NewDispenser(filename string, input io.Reader) Dispenser {
-	return Dispenser{
-		filename: filename,
-		tokens:   allTokens(input),
-		cursor:   -1,
-	}
-}
-
-// NewDispenserTokens returns a Dispenser filled with the given tokens.
-func NewDispenserTokens(filename string, tokens []token) Dispenser {
-	return Dispenser{
-		filename: filename,
-		tokens:   tokens,
-		cursor:   -1,
-	}
-}
-
-// Next loads the next token. Returns true if a token
-// was loaded; false otherwise. If false, all tokens
-// have already been consumed.
-func (d *Dispenser) Next() bool {
-	if d.cursor < len(d.tokens)-1 {
-		d.cursor++
-		return true
-	}
-	return false
-}
-
-// NextArg loads the next token if it is on the same
-// line. Returns true if a token was loaded; false
-// otherwise. If false, all tokens on the line have
-// been consumed.
-func (d *Dispenser) NextArg() bool {
-	if d.cursor < 0 {
-		d.cursor++
-		return true
-	}
-	if d.cursor >= len(d.tokens) {
-		return false
-	}
-	if d.cursor < len(d.tokens)-1 &&
-		(d.tokens[d.cursor].line+d.numLineBreaks(d.cursor) == d.tokens[d.cursor+1].line) {
-		d.cursor++
-		return true
-	}
-	return false
-}
-
-// NextLine loads the next token only if it is not on the same
-// line as the current token, and returns true if a token was
-// loaded; false otherwise. If false, there is not another token
-// or it is on the same line.
-func (d *Dispenser) NextLine() bool {
-	if d.cursor < 0 {
-		d.cursor++
-		return true
-	}
-	if d.cursor >= len(d.tokens) {
-		return false
-	}
-	if d.cursor < len(d.tokens)-1 &&
-		d.tokens[d.cursor].line+d.numLineBreaks(d.cursor) < d.tokens[d.cursor+1].line {
-		d.cursor++
-		return true
-	}
-	return false
-}
-
-// NextBlock can be used as the condition of a for loop
-// to load the next token as long as it opens a block or
-// is already in a block. It returns true if a token was
-// loaded, or false when the block's closing curly brace
-// was loaded and thus the block ended. Nested blocks are
-// not supported.
-func (d *Dispenser) NextBlock() bool {
-	if d.nesting > 0 {
-		d.Next()
-		if d.Val() == "}" {
-			d.nesting--
-			return false
-		}
-		return true
-	}
-	if !d.NextArg() { // block must open on same line
-		return false
-	}
-	if d.Val() != "{" {
-		d.cursor-- // roll back if not opening brace
-		return false
-	}
-	d.Next()
-	if d.Val() == "}" {
-		// Open and then closed right away
-		return false
-	}
-	d.nesting++
-	return true
-}
-
-// Val gets the text of the current token. If there is no token
-// loaded, it returns empty string.
-func (d *Dispenser) Val() string {
-	if d.cursor < 0 || d.cursor >= len(d.tokens) {
-		return ""
-	}
-	return d.tokens[d.cursor].text
-}
-
-// Line gets the line number of the current token. If there is no token
-// loaded, it returns 0.
-func (d *Dispenser) Line() int {
-	if d.cursor < 0 || d.cursor >= len(d.tokens) {
-		return 0
-	}
-	return d.tokens[d.cursor].line
-}
-
-// Args is a convenience function that loads the next arguments
-// (tokens on the same line) into an arbitrary number of strings
-// pointed to in targets. If there are fewer tokens available
-// than string pointers, the remaining strings will not be changed
-// and false will be returned. If there were enough tokens available
-// to fill the arguments, then true will be returned.
-func (d *Dispenser) Args(targets ...*string) bool {
-	enough := true
-	for i := 0; i < len(targets); i++ {
-		if !d.NextArg() {
-			enough = false
-			break
-		}
-		*targets[i] = d.Val()
-	}
-	return enough
-}
-
-// RemainingArgs loads any more arguments (tokens on the same line)
-// into a slice and returns them. Open curly brace tokens also indicate
-// the end of arguments, and the curly brace is not included in
-// the return value nor is it loaded.
-func (d *Dispenser) RemainingArgs() []string {
-	var args []string
-
-	for d.NextArg() {
-		if d.Val() == "{" {
-			d.cursor--
-			break
-		}
-		args = append(args, d.Val())
-	}
-
-	return args
-}
-
-// ArgErr returns an argument error, meaning that another
-// argument was expected but not found. In other words,
-// a line break or open curly brace was encountered instead of
-// an argument.
-func (d *Dispenser) ArgErr() error {
-	if d.Val() == "{" {
-		return d.Err("Unexpected token '{', expecting argument")
-	}
-	return d.Errf("Wrong argument count or unexpected line ending after '%s'", d.Val())
-}
-
-// SyntaxErr creates a generic syntax error which explains what was
-// found and what was expected.
-func (d *Dispenser) SyntaxErr(expected string) error {
-	msg := fmt.Sprintf("%s:%d - Syntax error: Unexpected token '%s', expecting '%s'", d.filename, d.Line(), d.Val(), expected)
-	return errors.New(msg)
-}
-
-// EofErr returns an EOF error, meaning that end of input
-// was found when another token was expected.
-func (d *Dispenser) EofErr() error {
-	return d.Errf("Unexpected EOF")
-}
-
-// Err generates a custom parse error with a message of msg.
-func (d *Dispenser) Err(msg string) error {
-	msg = fmt.Sprintf("%s:%d - Parse error: %s", d.filename, d.Line(), msg)
-	return errors.New(msg)
-}
-
-// Errf is like Err, but for formatted error messages
-func (d *Dispenser) Errf(format string, args ...interface{}) error {
-	return d.Err(fmt.Sprintf(format, args...))
-}
-
-// numLineBreaks counts how many line breaks are in the token
-// value given by the token index tknIdx. It returns 0 if the
-// token does not exist or there are no line breaks.
-func (d *Dispenser) numLineBreaks(tknIdx int) int {
-	if tknIdx < 0 || tknIdx >= len(d.tokens) {
-		return 0
-	}
-	return strings.Count(d.tokens[tknIdx].text, "\n")
-}

config/parse/lexer_test.go

@@ -1,165 +0,0 @@
-package parse
-
-import (
-	"strings"
-	"testing"
-)
-
-type lexerTestCase struct {
-	input    string
-	expected []token
-}
-
-func TestLexer(t *testing.T) {
-	testCases := []lexerTestCase{
-		{
-			input: `host:123`,
-			expected: []token{
-				{line: 1, text: "host:123"},
-			},
-		},
-		{
-			input: `host:123
-
-					directive`,
-			expected: []token{
-				{line: 1, text: "host:123"},
-				{line: 3, text: "directive"},
-			},
-		},
-		{
-			input: `host:123 {
-						directive
-					}`,
-			expected: []token{
-				{line: 1, text: "host:123"},
-				{line: 1, text: "{"},
-				{line: 2, text: "directive"},
-				{line: 3, text: "}"},
-			},
-		},
-		{
-			input: `host:123 { directive }`,
-			expected: []token{
-				{line: 1, text: "host:123"},
-				{line: 1, text: "{"},
-				{line: 1, text: "directive"},
-				{line: 1, text: "}"},
-			},
-		},
-		{
-			input: `host:123 {
-						#comment
-						directive
-						# comment
-						foobar # another comment
-					}`,
-			expected: []token{
-				{line: 1, text: "host:123"},
-				{line: 1, text: "{"},
-				{line: 3, text: "directive"},
-				{line: 5, text: "foobar"},
-				{line: 6, text: "}"},
-			},
-		},
-		{
-			input: `a "quoted value" b
-					foobar`,
-			expected: []token{
-				{line: 1, text: "a"},
-				{line: 1, text: "quoted value"},
-				{line: 1, text: "b"},
-				{line: 2, text: "foobar"},
-			},
-		},
-		{
-			input: `A "quoted \"value\" inside" B`,
-			expected: []token{
-				{line: 1, text: "A"},
-				{line: 1, text: `quoted "value" inside`},
-				{line: 1, text: "B"},
-			},
-		},
-		{
-			input: `"don't\escape"`,
-			expected: []token{
-				{line: 1, text: `don't\escape`},
-			},
-		},
-		{
-			input: `"don't\\escape"`,
-			expected: []token{
-				{line: 1, text: `don't\\escape`},
-			},
-		},
-		{
-			input: `A "quoted value with line
-					break inside" {
-						foobar
-					}`,
-			expected: []token{
-				{line: 1, text: "A"},
-				{line: 1, text: "quoted value with line\n\t\t\t\t\tbreak inside"},
-				{line: 2, text: "{"},
-				{line: 3, text: "foobar"},
-				{line: 4, text: "}"},
-			},
-		},
-		{
-			input: `"C:\php\php-cgi.exe"`,
-			expected: []token{
-				{line: 1, text: `C:\php\php-cgi.exe`},
-			},
-		},
-		{
-			input: `empty "" string`,
-			expected: []token{
-				{line: 1, text: `empty`},
-				{line: 1, text: ``},
-				{line: 1, text: `string`},
-			},
-		},
-		{
-			input: "skip those\r\nCR characters",
-			expected: []token{
-				{line: 1, text: "skip"},
-				{line: 1, text: "those"},
-				{line: 2, text: "CR"},
-				{line: 2, text: "characters"},
-			},
-		},
-	}
-
-	for i, testCase := range testCases {
-		actual := tokenize(testCase.input)
-		lexerCompare(t, i, testCase.expected, actual)
-	}
-}
-
-func tokenize(input string) (tokens []token) {
-	l := lexer{}
-	l.load(strings.NewReader(input))
-	for l.next() {
-		tokens = append(tokens, l.token)
-	}
-	return
-}
-
-func lexerCompare(t *testing.T, n int, expected, actual []token) {
-	if len(expected) != len(actual) {
-		t.Errorf("Test case %d: expected %d token(s) but got %d", n, len(expected), len(actual))
-	}
-
-	for i := 0; i < len(actual) && i < len(expected); i++ {
-		if actual[i].line != expected[i].line {
-			t.Errorf("Test case %d token %d ('%s'): expected line %d but was line %d",
-				n, i, expected[i].text, expected[i].line, actual[i].line)
-			break
-		}
-		if actual[i].text != expected[i].text {
-			t.Errorf("Test case %d token %d: expected text '%s' but was '%s'",
-				n, i, expected[i].text, actual[i].text)
-			break
-		}
-	}
-}

config/parse/parse.go

@@ -1,29 +0,0 @@
-// Package parse provides facilities for parsing configuration files.
-package parse
-
-import "io"
-
-// ServerBlocks parses the input just enough to organize tokens,
-// in order, by server block. No further parsing is performed.
-// Server blocks are returned in the order in which they appear.
-func ServerBlocks(filename string, input io.Reader) ([]serverBlock, error) {
-	p := parser{Dispenser: NewDispenser(filename, input)}
-	blocks, err := p.parseAll()
-	return blocks, err
-}
-
-// allTokens lexes the entire input, but does not parse it.
-// It returns all the tokens from the input, unstructured
-// and in order.
-func allTokens(input io.Reader) (tokens []token) {
-	l := new(lexer)
-	l.load(input)
-	for l.next() {
-		tokens = append(tokens, l.token)
-	}
-	return
-}
-
-// Set of directives that are valid (unordered). Populated
-// by config package's init function.
-var ValidDirectives = make(map[string]struct{})

config/parse/parse_test.go

@@ -1,22 +0,0 @@
-package parse
-
-import (
-	"strings"
-	"testing"
-)
-
-func TestAllTokens(t *testing.T) {
-	input := strings.NewReader("a b c\nd e")
-	expected := []string{"a", "b", "c", "d", "e"}
-	tokens := allTokens(input)
-
-	if len(tokens) != len(expected) {
-		t.Fatalf("Expected %d tokens, got %d", len(expected), len(tokens))
-	}
-
-	for i, val := range expected {
-		if tokens[i].text != val {
-			t.Errorf("Token %d should be '%s' but was '%s'", i, val, tokens[i].text)
-		}
-	}
-}

config/parse/parsing.go

@@ -1,310 +0,0 @@
-package parse
-
-import (
-	"net"
-	"os"
-	"strings"
-)
-
-type parser struct {
-	Dispenser
-	block multiServerBlock // current server block being parsed
-	eof   bool             // if we encounter a valid EOF in a hard place
-}
-
-func (p *parser) parseAll() ([]serverBlock, error) {
-	var blocks []serverBlock
-
-	for p.Next() {
-		err := p.parseOne()
-		if err != nil {
-			return blocks, err
-		}
-
-		// explode the multiServerBlock into multiple serverBlocks
-		for _, addr := range p.block.addresses {
-			blocks = append(blocks, serverBlock{
-				Host:   addr.host,
-				Port:   addr.port,
-				Tokens: p.block.tokens,
-			})
-		}
-	}
-
-	return blocks, nil
-}
-
-func (p *parser) parseOne() error {
-	p.block = multiServerBlock{tokens: make(map[string][]token)}
-
-	err := p.begin()
-	if err != nil {
-		return err
-	}
-
-	return nil
-}
-
-func (p *parser) begin() error {
-	if len(p.tokens) == 0 {
-		return nil
-	}
-
-	err := p.addresses()
-	if err != nil {
-		return err
-	}
-
-	if p.eof {
-		// this happens if the Caddyfile consists of only
-		// a line of addresses and nothing else
-		return nil
-	}
-
-	err = p.blockContents()
-	if err != nil {
-		return err
-	}
-
-	return nil
-}
-
-func (p *parser) addresses() error {
-	var expectingAnother bool
-
-	for {
-		tkn, startLine := p.Val(), p.Line()
-
-		// Open brace definitely indicates end of addresses
-		if tkn == "{" {
-			if expectingAnother {
-				return p.Errf("Expected another address but had '%s' - check for extra comma", tkn)
-			}
-			break
-		}
-
-		// Trailing comma indicates another address will follow, which
-		// may possibly be on the next line
-		if tkn[len(tkn)-1] == ',' {
-			tkn = tkn[:len(tkn)-1]
-			expectingAnother = true
-		} else {
-			expectingAnother = false // but we may still see another one on this line
-		}
-
-		// Parse and save this address
-		host, port, err := standardAddress(tkn)
-		if err != nil {
-			return err
-		}
-		p.block.addresses = append(p.block.addresses, address{host, port})
-
-		// Advance token and possibly break out of loop or return error
-		hasNext := p.Next()
-		if expectingAnother && !hasNext {
-			return p.EofErr()
-		}
-		if !expectingAnother && p.Line() > startLine {
-			break
-		}
-		if !hasNext {
-			p.eof = true
-			break // EOF
-		}
-	}
-
-	return nil
-}
-
-func (p *parser) blockContents() error {
-	errOpenCurlyBrace := p.openCurlyBrace()
-	if errOpenCurlyBrace != nil {
-		// single-server configs don't need curly braces
-		p.cursor--
-	}
-
-	err := p.directives()
-	if err != nil {
-		return err
-	}
-
-	// Only look for close curly brace if there was an opening
-	if errOpenCurlyBrace == nil {
-		err = p.closeCurlyBrace()
-		if err != nil {
-			return err
-		}
-	}
-
-	return nil
-}
-
-// directives parses through all the lines for directives
-// and it expects the next token to be the first
-// directive. It goes until EOF or closing curly brace
-// which ends the server block.
-func (p *parser) directives() error {
-	for p.Next() {
-		// end of server block
-		if p.Val() == "}" {
-			break
-		}
-
-		// special case: import directive replaces tokens during parse-time
-		if p.Val() == "import" {
-			err := p.doImport()
-			if err != nil {
-				return err
-			}
-			continue
-		}
-
-		// normal case: parse a directive on this line
-		if err := p.directive(); err != nil {
-			return err
-		}
-	}
-	return nil
-}
-
-// doImport swaps out the import directive and its argument
-// (a total of 2 tokens) with the tokens in the file specified.
-// When the function returns, the cursor is on the token before
-// where the import directive was. In other words, call Next()
-// to access the first token that was imported.
-func (p *parser) doImport() error {
-	if !p.NextArg() {
-		return p.ArgErr()
-	}
-	importFile := p.Val()
-	if p.NextArg() {
-		return p.Err("Import allows only one file to import")
-	}
-
-	file, err := os.Open(importFile)
-	if err != nil {
-		return p.Errf("Could not import %s - %v", importFile, err)
-	}
-	defer file.Close()
-	importedTokens := allTokens(file)
-
-	// Splice out the import directive and its argument (2 tokens total)
-	// and insert the imported tokens.
-	tokensBefore := p.tokens[:p.cursor-1]
-	tokensAfter := p.tokens[p.cursor+1:]
-	p.tokens = append(tokensBefore, append(importedTokens, tokensAfter...)...)
-	p.cursor -= 2
-
-	return nil
-}
-
-// directive collects tokens until the directive's scope
-// closes (either end of line or end of curly brace block).
-// It expects the currently-loaded token to be a directive
-// (or } that ends a server block). The collected tokens
-// are loaded into the current server block for later use
-// by directive setup functions.
-func (p *parser) directive() error {
-	dir := p.Val()
-	line := p.Line()
-	nesting := 0
-
-	if _, ok := ValidDirectives[dir]; !ok {
-		return p.Errf("Unknown directive '%s'", dir)
-	}
-
-	// The directive itself is appended as a relevant token
-	p.block.tokens[dir] = append(p.block.tokens[dir], p.tokens[p.cursor])
-
-	for p.Next() {
-		if p.Val() == "{" {
-			nesting++
-		} else if p.Line()+p.numLineBreaks(p.cursor) > line && nesting == 0 {
-			p.cursor-- // read too far
-			break
-		} else if p.Val() == "}" && nesting > 0 {
-			nesting--
-		} else if p.Val() == "}" && nesting == 0 {
-			return p.Err("Unexpected '}' because no matching opening brace")
-		}
-		p.block.tokens[dir] = append(p.block.tokens[dir], p.tokens[p.cursor])
-	}
-
-	if nesting > 0 {
-		return p.EofErr()
-	}
-	return nil
-}
-
-// openCurlyBrace expects the current token to be an
-// opening curly brace. This acts like an assertion
-// because it returns an error if the token is not
-// a opening curly brace. It does not advance the token.
-func (p *parser) openCurlyBrace() error {
-	if p.Val() != "{" {
-		return p.SyntaxErr("{")
-	}
-	return nil
-}
-
-// closeCurlyBrace expects the current token to be
-// a closing curly brace. This acts like an assertion
-// because it returns an error if the token is not
-// a closing curly brace. It does not advance the token.
-func (p *parser) closeCurlyBrace() error {
-	if p.Val() != "}" {
-		return p.SyntaxErr("}")
-	}
-	return nil
-}
-
-// standardAddress turns the accepted host and port patterns
-// into a format accepted by net.Dial.
-func standardAddress(str string) (host, port string, err error) {
-	var schemePort, splitPort string
-
-	if strings.HasPrefix(str, "https://") {
-		schemePort = "https"
-		str = str[8:]
-	} else if strings.HasPrefix(str, "http://") {
-		schemePort = "http"
-		str = str[7:]
-	}
-
-	host, splitPort, err = net.SplitHostPort(str)
-	if err != nil {
-		host, splitPort, err = net.SplitHostPort(str + ":") // tack on empty port
-	}
-	if err != nil {
-		// ¯\_(ツ)_/¯
-		host = str
-	}
-
-	if splitPort != "" {
-		port = splitPort
-	} else {
-		port = schemePort
-	}
-
-	return
-}
-
-type (
-	// serverBlock stores tokens by directive name for a
-	// single host:port (address)
-	serverBlock struct {
-		Host, Port string
-		Tokens     map[string][]token // directive name to tokens (including directive)
-	}
-
-	// multiServerBlock is the same as serverBlock but for
-	// multiple addresses that share the same tokens
-	multiServerBlock struct {
-		addresses []address
-		tokens    map[string][]token
-	}
-
-	address struct {
-		host, port string
-	}
-)

config/parse/parsing_test.go

@@ -1,365 +0,0 @@
-package parse
-
-import (
-	"reflect"
-	"strings"
-	"testing"
-)
-
-func TestStandardAddress(t *testing.T) {
-	for i, test := range []struct {
-		input      string
-		host, port string
-		shouldErr  bool
-	}{
-		{`localhost`, "localhost", "", false},
-		{`localhost:1234`, "localhost", "1234", false},
-		{`localhost:`, "localhost", "", false},
-		{`0.0.0.0`, "0.0.0.0", "", false},
-		{`127.0.0.1:1234`, "127.0.0.1", "1234", false},
-		{`:1234`, "", "1234", false},
-		{`[::1]`, "::1", "", false},
-		{`[::1]:1234`, "::1", "1234", false},
-		{`:`, "", "", false},
-		{`localhost:http`, "localhost", "http", false},
-		{`localhost:https`, "localhost", "https", false},
-		{`:http`, "", "http", false},
-		{`:https`, "", "https", false},
-		{`http://localhost`, "localhost", "http", false},
-		{`https://localhost`, "localhost", "https", false},
-		{`http://127.0.0.1`, "127.0.0.1", "http", false},
-		{`https://127.0.0.1`, "127.0.0.1", "https", false},
-		{`http://[::1]`, "::1", "http", false},
-		{`http://localhost:1234`, "localhost", "1234", false},
-		{`https://127.0.0.1:1234`, "127.0.0.1", "1234", false},
-		{`http://[::1]:1234`, "::1", "1234", false},
-		{``, "", "", false},
-		{`::1`, "::1", "", true},
-		{`localhost::`, "localhost::", "", true},
-		{`#$%@`, "#$%@", "", true},
-	} {
-		host, port, err := standardAddress(test.input)
-
-		if err != nil && !test.shouldErr {
-			t.Errorf("Test %d: Expected no error, but had error: %v", i, err)
-		}
-		if err == nil && test.shouldErr {
-			t.Errorf("Test %d: Expected error, but had none", i)
-		}
-
-		if host != test.host {
-			t.Errorf("Test %d: Expected host '%s', got '%s'", i, test.host, host)
-		}
-
-		if port != test.port {
-			t.Errorf("Test %d: Expected port '%s', got '%s'", i, test.port, port)
-		}
-	}
-}
-
-func TestParseOne(t *testing.T) {
-	setupParseTests()
-
-	testParseOne := func(input string) (multiServerBlock, error) {
-		p := testParser(input)
-		p.Next()
-		err := p.parseOne()
-		return p.block, err
-	}
-
-	for i, test := range []struct {
-		input     string
-		shouldErr bool
-		addresses []address
-		tokens    map[string]int // map of directive name to number of tokens expected
-	}{
-		{`localhost`, false, []address{
-			{"localhost", ""},
-		}, map[string]int{}},
-
-		{`localhost
-		  dir1`, false, []address{
-			{"localhost", ""},
-		}, map[string]int{
-			"dir1": 1,
-		}},
-
-		{`localhost:1234
-		  dir1 foo bar`, false, []address{
-			{"localhost", "1234"},
-		}, map[string]int{
-			"dir1": 3,
-		}},
-
-		{`localhost {
-		    dir1
-		  }`, false, []address{
-			{"localhost", ""},
-		}, map[string]int{
-			"dir1": 1,
-		}},
-
-		{`localhost:1234 {
-		    dir1 foo bar
-		    dir2
-		  }`, false, []address{
-			{"localhost", "1234"},
-		}, map[string]int{
-			"dir1": 3,
-			"dir2": 1,
-		}},
-
-		{`http://localhost https://localhost
-		  dir1 foo bar`, false, []address{
-			{"localhost", "http"},
-			{"localhost", "https"},
-		}, map[string]int{
-			"dir1": 3,
-		}},
-
-		{`http://localhost https://localhost {
-		    dir1 foo bar
-		  }`, false, []address{
-			{"localhost", "http"},
-			{"localhost", "https"},
-		}, map[string]int{
-			"dir1": 3,
-		}},
-
-		{`http://localhost, https://localhost {
-		    dir1 foo bar
-		  }`, false, []address{
-			{"localhost", "http"},
-			{"localhost", "https"},
-		}, map[string]int{
-			"dir1": 3,
-		}},
-
-		{`http://localhost, {
-		  }`, true, []address{
-			{"localhost", "http"},
-		}, map[string]int{}},
-
-		{`host1:80, http://host2.com
-		  dir1 foo bar
-		  dir2 baz`, false, []address{
-			{"host1", "80"},
-			{"host2.com", "http"},
-		}, map[string]int{
-			"dir1": 3,
-			"dir2": 2,
-		}},
-
-		{`http://host1.com,
-		  http://host2.com,
-		  https://host3.com`, false, []address{
-			{"host1.com", "http"},
-			{"host2.com", "http"},
-			{"host3.com", "https"},
-		}, map[string]int{}},
-
-		{`http://host1.com:1234, https://host2.com
-		  dir1 foo {
-		    bar baz
-		  }
-		  dir2`, false, []address{
-			{"host1.com", "1234"},
-			{"host2.com", "https"},
-		}, map[string]int{
-			"dir1": 6,
-			"dir2": 1,
-		}},
-
-		{`127.0.0.1
-		  dir1 {
-		    bar baz
-		  }
-		  dir2 {
-		    foo bar
-		  }`, false, []address{
-			{"127.0.0.1", ""},
-		}, map[string]int{
-			"dir1": 5,
-			"dir2": 5,
-		}},
-
-		{`127.0.0.1
-		  unknown_directive`, true, []address{
-			{"127.0.0.1", ""},
-		}, map[string]int{}},
-
-		{`localhost
-		  dir1 {
-		    foo`, true, []address{
-			{"localhost", ""},
-		}, map[string]int{
-			"dir1": 3,
-		}},
-
-		{`localhost
-		  dir1 {
-		  }`, false, []address{
-			{"localhost", ""},
-		}, map[string]int{
-			"dir1": 3,
-		}},
-
-		{`localhost
-		  dir1 {
-		    nested {
-		      foo
-		    }
-		  }
-		  dir2 foo bar`, false, []address{
-			{"localhost", ""},
-		}, map[string]int{
-			"dir1": 7,
-			"dir2": 3,
-		}},
-
-		{``, false, []address{}, map[string]int{}},
-	} {
-		result, err := testParseOne(test.input)
-
-		if test.shouldErr && err == nil {
-			t.Errorf("Test %d: Expected an error, but didn't get one", i)
-		}
-		if !test.shouldErr && err != nil {
-			t.Errorf("Test %d: Expected no error, but got: %v", i, err)
-		}
-
-		if len(result.addresses) != len(test.addresses) {
-			t.Errorf("Test %d: Expected %d addresses, got %d",
-				i, len(test.addresses), len(result.addresses))
-			continue
-		}
-		for j, addr := range result.addresses {
-			if addr.host != test.addresses[j].host {
-				t.Errorf("Test %d, address %d: Expected host to be '%s', but was '%s'",
-					i, j, test.addresses[j].host, addr.host)
-			}
-			if addr.port != test.addresses[j].port {
-				t.Errorf("Test %d, address %d: Expected port to be '%s', but was '%s'",
-					i, j, test.addresses[j].port, addr.port)
-			}
-		}
-
-		if len(result.tokens) != len(test.tokens) {
-			t.Errorf("Test %d: Expected %d directives, had %d",
-				i, len(test.tokens), len(result.tokens))
-			continue
-		}
-		for directive, tokens := range result.tokens {
-			if len(tokens) != test.tokens[directive] {
-				t.Errorf("Test %d, directive '%s': Expected %d tokens, counted %d",
-					i, directive, test.tokens[directive], len(tokens))
-				continue
-			}
-		}
-	}
-}
-
-func TestParseAll(t *testing.T) {
-	setupParseTests()
-
-	for i, test := range []struct {
-		input     string
-		shouldErr bool
-		addresses []address // one per expected server block, in order
-	}{
-		{`localhost`, false, []address{
-			{"localhost", ""},
-		}},
-
-		{`localhost:1234`, false, []address{
-			{"localhost", "1234"},
-		}},
-
-		{`localhost:1234 {
-		  }
-		  localhost:2015 {
-		  }`, false, []address{
-			{"localhost", "1234"},
-			{"localhost", "2015"},
-		}},
-
-		{`localhost:1234, http://host2`, false, []address{
-			{"localhost", "1234"},
-			{"host2", "http"},
-		}},
-
-		{`localhost:1234, http://host2,`, true, []address{}},
-
-		{`http://host1.com, http://host2.com {
-		  }
-		  https://host3.com, https://host4.com {
-		  }`, false, []address{
-			{"host1.com", "http"},
-			{"host2.com", "http"},
-			{"host3.com", "https"},
-			{"host4.com", "https"},
-		}},
-	} {
-		p := testParser(test.input)
-		blocks, err := p.parseAll()
-
-		if test.shouldErr && err == nil {
-			t.Errorf("Test %d: Expected an error, but didn't get one", i)
-		}
-		if !test.shouldErr && err != nil {
-			t.Errorf("Test %d: Expected no error, but got: %v", i, err)
-		}
-
-		if len(blocks) != len(test.addresses) {
-			t.Errorf("Test %d: Expected %d server blocks, got %d",
-				i, len(test.addresses), len(blocks))
-			continue
-		}
-		for j, block := range blocks {
-			if block.Host != test.addresses[j].host {
-				t.Errorf("Test %d, block %d: Expected host to be '%s', but was '%s'",
-					i, j, test.addresses[j].host, block.Host)
-			}
-			if block.Port != test.addresses[j].port {
-				t.Errorf("Test %d, block %d: Expected port to be '%s', but was '%s'",
-					i, j, test.addresses[j].port, block.Port)
-			}
-		}
-	}
-
-	// Exploding the server blocks that have more than one address should replicate/share tokens
-	p := testParser(`host1 {
-	                   dir1 foo bar
-	                 }
-
-	                 host2, host3 {
-	                   dir2 foo bar
-	                   dir3 foo {
-	                     bar
-	                   }
-	                 }`)
-	blocks, err := p.parseAll()
-	if err != nil {
-		t.Fatalf("Expected there to not be an error, but there was: %v", err)
-	}
-
-	if !reflect.DeepEqual(blocks[1].Tokens, blocks[2].Tokens) {
-		t.Errorf("Expected host2 and host3 to have same tokens, but they didn't.\nhost2 Block: %v\nhost3 Block: %v",
-			blocks[1].Tokens, blocks[2].Tokens)
-	}
-}
-
-func setupParseTests() {
-	// Set up some bogus directives for testing
-	ValidDirectives = map[string]struct{}{
-		"dir1": struct{}{},
-		"dir2": struct{}{},
-		"dir3": struct{}{},
-	}
-}
-
-func testParser(input string) parser {
-	buf := strings.NewReader(input)
-	p := parser{Dispenser: NewDispenser("Test", buf)}
-	return p
-}

config/setup/basicauth.go

@@ -1,53 +0,0 @@
-package setup
-
-import (
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/basicauth"
-)
-
-// BasicAuth configures a new BasicAuth middleware instance.
-func BasicAuth(c *Controller) (middleware.Middleware, error) {
-	rules, err := basicAuthParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	basic := basicauth.BasicAuth{Rules: rules}
-
-	return func(next middleware.Handler) middleware.Handler {
-		basic.Next = next
-		return basic
-	}, nil
-}
-
-func basicAuthParse(c *Controller) ([]basicauth.Rule, error) {
-	var rules []basicauth.Rule
-
-	for c.Next() {
-		var rule basicauth.Rule
-
-		args := c.RemainingArgs()
-
-		switch len(args) {
-		case 2:
-			rule.Username = args[0]
-			rule.Password = args[1]
-			for c.NextBlock() {
-				rule.Resources = append(rule.Resources, c.Val())
-				if c.NextArg() {
-					return rules, c.Errf("Expecting only one resource per line (extra '%s')", c.Val())
-				}
-			}
-		case 3:
-			rule.Resources = append(rule.Resources, args[0])
-			rule.Username = args[1]
-			rule.Password = args[2]
-		default:
-			return rules, c.ArgErr()
-		}
-
-		rules = append(rules, rule)
-	}
-
-	return rules, nil
-}

config/setup/basicauth_test.go

@@ -1,100 +0,0 @@
-package setup
-
-import (
-	"fmt"
-	"testing"
-
-	"github.com/mholt/caddy/middleware/basicauth"
-)
-
-func TestBasicAuth(t *testing.T) {
-	c := newTestController(`basicauth user pwd`)
-
-	mid, err := BasicAuth(c)
-	if err != nil {
-		t.Errorf("Expected no errors, but got: %v", err)
-	}
-	if mid == nil {
-		t.Fatal("Expected middleware, was nil instead")
-	}
-
-	handler := mid(emptyNext)
-	myHandler, ok := handler.(basicauth.BasicAuth)
-	if !ok {
-		t.Fatalf("Expected handler to be type BasicAuth, got: %#v", handler)
-	}
-
-	if !sameNext(myHandler.Next, emptyNext) {
-		t.Error("'Next' field of handler was not set properly")
-	}
-}
-
-func TestBasicAuthParse(t *testing.T) {
-	tests := []struct {
-		input     string
-		shouldErr bool
-		expected  []basicauth.Rule
-	}{
-		{`basicauth user pwd`, false, []basicauth.Rule{
-			{Username: "user", Password: "pwd"},
-		}},
-		{`basicauth user pwd {
-		}`, false, []basicauth.Rule{
-			{Username: "user", Password: "pwd"},
-		}},
-		{`basicauth user pwd {
-			/resource1
-			/resource2
-		}`, false, []basicauth.Rule{
-			{Username: "user", Password: "pwd", Resources: []string{"/resource1", "/resource2"}},
-		}},
-		{`basicauth /resource user pwd`, false, []basicauth.Rule{
-			{Username: "user", Password: "pwd", Resources: []string{"/resource"}},
-		}},
-		{`basicauth /res1 user1 pwd1
-		  basicauth /res2 user2 pwd2`, false, []basicauth.Rule{
-			{Username: "user1", Password: "pwd1", Resources: []string{"/res1"}},
-			{Username: "user2", Password: "pwd2", Resources: []string{"/res2"}},
-		}},
-		{`basicauth user`, true, []basicauth.Rule{}},
-		{`basicauth`, true, []basicauth.Rule{}},
-		{`basicauth /resource user pwd asdf`, true, []basicauth.Rule{}},
-	}
-
-	for i, test := range tests {
-		c := newTestController(test.input)
-		actual, err := basicAuthParse(c)
-
-		if err == nil && test.shouldErr {
-			t.Errorf("Test %d didn't error, but it should have", i)
-		} else if err != nil && !test.shouldErr {
-			t.Errorf("Test %d errored, but it shouldn't have; got '%v'", i, err)
-		}
-
-		if len(actual) != len(test.expected) {
-			t.Fatalf("Test %d expected %d rules, but got %d",
-				i, len(test.expected), len(actual))
-		}
-
-		for j, expectedRule := range test.expected {
-			actualRule := actual[j]
-
-			if actualRule.Username != expectedRule.Username {
-				t.Errorf("Test %d, rule %d: Expected username '%s', got '%s'",
-					i, j, expectedRule.Username, actualRule.Username)
-			}
-
-			if actualRule.Password != expectedRule.Password {
-				t.Errorf("Test %d, rule %d: Expected password '%s', got '%s'",
-					i, j, expectedRule.Password, actualRule.Password)
-			}
-
-			expectedRes := fmt.Sprintf("%v", expectedRule.Resources)
-			actualRes := fmt.Sprintf("%v", actualRule.Resources)
-			if actualRes != expectedRes {
-				t.Errorf("Test %d, rule %d: Expected resource list %s, but got %s",
-					i, j, expectedRes, actualRes)
-			}
-		}
-	}
-}

config/setup/bindhost.go

@@ -1,12 +0,0 @@
-package setup
-
-import "github.com/mholt/caddy/middleware"
-
-func BindHost(c *Controller) (middleware.Middleware, error) {
-	for c.Next() {
-		if !c.Args(&c.BindHost) {
-			return nil, c.ArgErr()
-		}
-	}
-	return nil, nil
-}

config/setup/browse.go

@@ -1,222 +0,0 @@
-package setup
-
-import (
-	"fmt"
-	"html/template"
-	"io/ioutil"
-
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/browse"
-)
-
-// Browse configures a new Browse middleware instance.
-func Browse(c *Controller) (middleware.Middleware, error) {
-	configs, err := browseParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	browse := browse.Browse{
-		Root:    c.Root,
-		Configs: configs,
-	}
-
-	return func(next middleware.Handler) middleware.Handler {
-		browse.Next = next
-		return browse
-	}, nil
-}
-
-func browseParse(c *Controller) ([]browse.Config, error) {
-	var configs []browse.Config
-
-	appendCfg := func(bc browse.Config) error {
-		for _, c := range configs {
-			if c.PathScope == bc.PathScope {
-				return fmt.Errorf("Duplicate browsing config for %s", c.PathScope)
-			}
-		}
-		configs = append(configs, bc)
-		return nil
-	}
-
-	for c.Next() {
-		var bc browse.Config
-
-		// First argument is directory to allow browsing; default is site root
-		if c.NextArg() {
-			bc.PathScope = c.Val()
-		} else {
-			bc.PathScope = "/"
-		}
-
-		// Second argument would be the template file to use
-		var tplText string
-		if c.NextArg() {
-			tplBytes, err := ioutil.ReadFile(c.Val())
-			if err != nil {
-				return configs, err
-			}
-			tplText = string(tplBytes)
-		} else {
-			tplText = defaultTemplate
-		}
-
-		// Build the template
-		tpl, err := template.New("listing").Parse(tplText)
-		if err != nil {
-			return configs, err
-		}
-		bc.Template = tpl
-
-		// Save configuration
-		err = appendCfg(bc)
-		if err != nil {
-			return configs, err
-		}
-	}
-
-	return configs, nil
-}
-
-// The default template to use when serving up directory listings
-const defaultTemplate = `<!DOCTYPE html>
-<html>
-	<head>
-		<title>{{.Name}}</title>
-		<meta charset="utf-8">
-<style>
-* { padding: 0; margin: 0; }
-
-body {
-	padding: 1% 2%;
-	font: 16px Arial;
-}
-
-header {
-	font-size: 45px;
-	padding: 25px;
-}
-
-header a {
-	text-decoration: none;
-	color: inherit;
-}
-
-header .up {
-	display: inline-block;
-	height: 50px;
-	width: 50px;
-	text-align: center;
-	margin-right: 20px;
-}
-
-header a.up:hover {
-	background: #000;
-	color: #FFF;
-}
-
-h1 {
-	font-size: 30px;
-	display: inline;
-}
-
-table {
-	border: 0;
-	border-collapse: collapse;
-	max-width: 750px;
-	margin: 0 auto;
-}
-
-th,
-td {
-	padding: 4px 20px;
-	vertical-align: middle;
-	line-height: 1.5em; /* emoji are kind of odd heights */
-}
-
-th {
-	text-align: left;
-}
-
-@media (max-width: 700px) {
-	.hideable {
-		display: none;
-	}
-
-	body {
-		padding: 0;
-	}
-
-	header,
-	header h1 {
-		font-size: 16px;
-	}
-
-	header {
-		position: fixed;
-		top: 0;
-		width: 100%;
-		background: #333;
-		color: #FFF;
-		padding: 15px;
-		text-align: center;
-	}
-
-	header .up {
-		height: auto;
-		width: auto;
-		display: none;
-	}
-
-	header a.up {
-		display: inline-block;
-		position: absolute;
-		left: 0;
-		top: 0;
-		width: 40px;
-		height: 48px;
-		font-size: 35px;
-	}
-
-	header h1 {
-		font-weight: normal;
-	}
-
-	main {
-		margin-top: 70px;
-	}
-}
-</style>
-	</head>
-	<body>
-		<header>
-			{{if .CanGoUp}}
-			<a href=".." class="up" title="Up one level">&#11025;</a>
-			{{else}}
-			<div class="up">&nbsp;</div>
-			{{end}}
-
-			<h1>{{.Path}}</h1>
-		</header>
-		<main>
-			<table>
-				<tr>
-					<th>Name</th>
-					<th>Size</th>
-					<th class="hideable">Modified</th>
-				</tr>
-				{{range .Items}}
-				<tr>
-					<td>
-						{{if .IsDir}}&#128194;{{else}}&#128196;{{end}}
-						<a href="{{.URL}}">{{.Name}}</a>
-					</td>
-					<td>{{.HumanSize}}</td>
-					<td class="hideable">{{.HumanModTime "01/02/2006 3:04:05 PM -0700"}}</td>
-				</tr>
-				{{end}}
-			</table>
-		</main>
-	</body>
-</html>`

config/setup/controller.go

@@ -1,11 +0,0 @@
-package setup
-
-import (
-	"github.com/mholt/caddy/config/parse"
-	"github.com/mholt/caddy/server"
-)
-
-type Controller struct {
-	*server.Config
-	parse.Dispenser
-}

config/setup/controller_test.go

@@ -1,32 +0,0 @@
-package setup
-
-import (
-	"fmt"
-	"net/http"
-	"strings"
-
-	"github.com/mholt/caddy/config/parse"
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/server"
-)
-
-// newTestController creates a new *Controller for
-// the input specified, with a filename of "Testfile"
-func newTestController(input string) *Controller {
-	return &Controller{
-		Config:    &server.Config{},
-		Dispenser: parse.NewDispenser("Testfile", strings.NewReader(input)),
-	}
-}
-
-// emptyNext is a no-op function that can be passed into
-// middleware.Middleware functions so that the assignment
-// to the Next field of the Handler can be tested.
-var emptyNext = middleware.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
-	return 0, nil
-})
-
-// sameNext does a pointer comparison between next1 and next2.
-func sameNext(next1, next2 middleware.Handler) bool {
-	return fmt.Sprintf("%p", next1) == fmt.Sprintf("%p", next2)
-}

config/setup/errors.go

@@ -1,103 +0,0 @@
-package setup
-
-import (
-	"log"
-	"os"
-	"path"
-	"strconv"
-
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/errors"
-)
-
-// Errors configures a new gzip middleware instance.
-func Errors(c *Controller) (middleware.Middleware, error) {
-	handler, err := errorsParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	// Open the log file for writing when the server starts
-	c.Startup = append(c.Startup, func() error {
-		var err error
-		var file *os.File
-
-		if handler.LogFile == "stdout" {
-			file = os.Stdout
-		} else if handler.LogFile == "stderr" {
-			file = os.Stderr
-		} else if handler.LogFile != "" {
-			file, err = os.OpenFile(handler.LogFile, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
-			if err != nil {
-				return err
-			}
-		}
-
-		handler.Log = log.New(file, "", 0)
-		return nil
-	})
-
-	return func(next middleware.Handler) middleware.Handler {
-		handler.Next = next
-		return handler
-	}, nil
-}
-
-func errorsParse(c *Controller) (*errors.ErrorHandler, error) {
-	// Very important that we make a pointer because the Startup
-	// function that opens the log file must have access to the
-	// same instance of the handler, not a copy.
-	handler := &errors.ErrorHandler{ErrorPages: make(map[int]string)}
-
-	optionalBlock := func() (bool, error) {
-		var hadBlock bool
-
-		for c.NextBlock() {
-			hadBlock = true
-
-			what := c.Val()
-			if !c.NextArg() {
-				return hadBlock, c.ArgErr()
-			}
-			where := c.Val()
-
-			if what == "log" {
-				handler.LogFile = where
-			} else {
-				// Error page; ensure it exists
-				where = path.Join(c.Root, where)
-				f, err := os.Open(where)
-				if err != nil {
-					return hadBlock, c.Err("Unable to open error page '" + where + "': " + err.Error())
-				}
-				f.Close()
-
-				whatInt, err := strconv.Atoi(what)
-				if err != nil {
-					return hadBlock, c.Err("Expecting a numeric status code, got '" + what + "'")
-				}
-				handler.ErrorPages[whatInt] = where
-			}
-		}
-		return hadBlock, nil
-	}
-
-	for c.Next() {
-		// Configuration may be in a block
-		hadBlock, err := optionalBlock()
-		if err != nil {
-			return handler, err
-		}
-
-		// Otherwise, the only argument would be an error log file name
-		if !hadBlock {
-			if c.NextArg() {
-				handler.LogFile = c.Val()
-			} else {
-				handler.LogFile = errors.DefaultLogFilename
-			}
-		}
-	}
-
-	return handler, nil
-}

config/setup/ext.go

@@ -1,54 +0,0 @@
-package setup
-
-import (
-	"os"
-
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/extensions"
-)
-
-// Ext configures a new instance of 'extensions' middleware for clean URLs.
-func Ext(c *Controller) (middleware.Middleware, error) {
-	root := c.Root
-
-	exts, err := extParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	return func(next middleware.Handler) middleware.Handler {
-		return extensions.Ext{
-			Next:       next,
-			Extensions: exts,
-			Root:       root,
-		}
-	}, nil
-}
-
-// extParse sets up an instance of extension middleware
-// from a middleware controller and returns a list of extensions.
-func extParse(c *Controller) ([]string, error) {
-	var exts []string
-
-	for c.Next() {
-		// At least one extension is required
-		if !c.NextArg() {
-			return exts, c.ArgErr()
-		}
-		exts = append(exts, c.Val())
-
-		// Tack on any other extensions that may have been listed
-		exts = append(exts, c.RemainingArgs()...)
-	}
-
-	return exts, nil
-}
-
-// resourceExists returns true if the file specified at
-// root + path exists; false otherwise.
-func resourceExists(root, path string) bool {
-	_, err := os.Stat(root + path)
-	// technically we should use os.IsNotExist(err)
-	// but we don't handle any other kinds of errors anyway
-	return err == nil
-}

config/setup/fastcgi.go

@@ -1,110 +0,0 @@
-package setup
-
-import (
-	"errors"
-	"net/http"
-	"path/filepath"
-
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/fastcgi"
-)
-
-// FastCGI configures a new FastCGI middleware instance.
-func FastCGI(c *Controller) (middleware.Middleware, error) {
-	absRoot, err := filepath.Abs(c.Root)
-	if err != nil {
-		return nil, err
-	}
-
-	rules, err := fastcgiParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	return func(next middleware.Handler) middleware.Handler {
-		return fastcgi.Handler{
-			Next:            next,
-			Rules:           rules,
-			Root:            c.Root,
-			AbsRoot:         absRoot,
-			FileSys:         http.Dir(c.Root),
-			SoftwareName:    c.AppName,
-			SoftwareVersion: c.AppVersion,
-			ServerName:      c.Host,
-			ServerPort:      c.Port,
-		}
-	}, nil
-}
-
-func fastcgiParse(c *Controller) ([]fastcgi.Rule, error) {
-	var rules []fastcgi.Rule
-
-	for c.Next() {
-		var rule fastcgi.Rule
-
-		args := c.RemainingArgs()
-
-		switch len(args) {
-		case 0:
-			return rules, c.ArgErr()
-		case 1:
-			rule.Path = "/"
-			rule.Address = args[0]
-		case 2:
-			rule.Path = args[0]
-			rule.Address = args[1]
-		case 3:
-			rule.Path = args[0]
-			rule.Address = args[1]
-			err := fastcgiPreset(args[2], &rule)
-			if err != nil {
-				return rules, c.Err("Invalid fastcgi rule preset '" + args[2] + "'")
-			}
-		}
-
-		for c.NextBlock() {
-			switch c.Val() {
-			case "ext":
-				if !c.NextArg() {
-					return rules, c.ArgErr()
-				}
-				rule.Ext = c.Val()
-			case "split":
-				if !c.NextArg() {
-					return rules, c.ArgErr()
-				}
-				rule.SplitPath = c.Val()
-			case "index":
-				args := c.RemainingArgs()
-				if len(args) == 0 {
-					return rules, c.ArgErr()
-				}
-				rule.IndexFiles = args
-			case "env":
-				envArgs := c.RemainingArgs()
-				if len(envArgs) < 2 {
-					return rules, c.ArgErr()
-				}
-				rule.EnvVars = append(rule.EnvVars, [2]string{envArgs[0], envArgs[1]})
-			}
-		}
-
-		rules = append(rules, rule)
-	}
-
-	return rules, nil
-}
-
-// fastcgiPreset configures rule according to name. It returns an error if
-// name is not a recognized preset name.
-func fastcgiPreset(name string, rule *fastcgi.Rule) error {
-	switch name {
-	case "php":
-		rule.Ext = ".php"
-		rule.SplitPath = ".php"
-		rule.IndexFiles = []string{"index.php"}
-	default:
-		return errors.New(name + " is not a valid preset name")
-	}
-	return nil
-}

config/setup/git.go

@@ -1,193 +0,0 @@
-package setup
-
-import (
-	"fmt"
-	"log"
-	"net/url"
-	"path/filepath"
-	"runtime"
-	"strconv"
-	"strings"
-	"time"
-
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/git"
-)
-
-// Git configures a new Git service routine.
-func Git(c *Controller) (middleware.Middleware, error) {
-	repo, err := gitParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	c.Startup = append(c.Startup, func() error {
-		// Startup functions are blocking; start
-		// service routine in background
-		go func() {
-			for {
-				time.Sleep(repo.Interval)
-
-				err := repo.Pull()
-				if err != nil {
-					if git.Logger == nil {
-						log.Println(err)
-					} else {
-						git.Logger.Println(err)
-					}
-				}
-			}
-		}()
-
-		// Do a pull right away to return error
-		return repo.Pull()
-	})
-
-	return nil, err
-}
-
-func gitParse(c *Controller) (*git.Repo, error) {
-	repo := &git.Repo{Branch: "master", Interval: git.DefaultInterval, Path: c.Root}
-
-	for c.Next() {
-		args := c.RemainingArgs()
-
-		switch len(args) {
-		case 2:
-			repo.Path = filepath.Clean(c.Root + string(filepath.Separator) + args[1])
-			fallthrough
-		case 1:
-			repo.URL = args[0]
-		}
-
-		for c.NextBlock() {
-			switch c.Val() {
-			case "repo":
-				if !c.NextArg() {
-					return nil, c.ArgErr()
-				}
-				repo.URL = c.Val()
-			case "path":
-				if !c.NextArg() {
-					return nil, c.ArgErr()
-				}
-				repo.Path = filepath.Clean(c.Root + string(filepath.Separator) + c.Val())
-			case "branch":
-				if !c.NextArg() {
-					return nil, c.ArgErr()
-				}
-				repo.Branch = c.Val()
-			case "key":
-				if !c.NextArg() {
-					return nil, c.ArgErr()
-				}
-				repo.KeyPath = c.Val()
-			case "interval":
-				if !c.NextArg() {
-					return nil, c.ArgErr()
-				}
-				t, _ := strconv.Atoi(c.Val())
-				if t > 0 {
-					repo.Interval = time.Duration(t) * time.Second
-				}
-			case "then":
-				thenArgs := c.RemainingArgs()
-				if len(thenArgs) == 0 {
-					return nil, c.ArgErr()
-				}
-				repo.Then = strings.Join(thenArgs, " ")
-			default:
-				return nil, c.ArgErr()
-			}
-		}
-	}
-
-	// if repo is not specified, return error
-	if repo.URL == "" {
-		return nil, c.ArgErr()
-	}
-
-	// if private key is not specified, convert repository URL to https
-	// to avoid ssh authentication
-	// else validate git URL
-	// Note: private key support not yet available on Windows
-	var err error
-	if repo.KeyPath == "" {
-		repo.URL, repo.Host, err = sanitizeHTTP(repo.URL)
-	} else {
-		repo.URL, repo.Host, err = sanitizeGit(repo.URL)
-		// TODO add Windows support for private repos
-		if runtime.GOOS == "windows" {
-			return nil, fmt.Errorf("Private repository not yet supported on Windows")
-		}
-	}
-
-	if err != nil {
-		return nil, err
-	}
-
-	// validate git requirements
-	if err = git.Init(); err != nil {
-		return nil, err
-	}
-
-	return repo, repo.Prepare()
-}
-
-// sanitizeHTTP cleans up repository URL and converts to https format
-// if currently in ssh format.
-// Returns sanitized url, hostName (e.g. github.com, bitbucket.com)
-// and possible error
-func sanitizeHTTP(repoURL string) (string, string, error) {
-	url, err := url.Parse(repoURL)
-	if err != nil {
-		return "", "", err
-	}
-
-	if url.Host == "" && strings.HasPrefix(url.Path, "git@") {
-		url.Path = url.Path[len("git@"):]
-		i := strings.Index(url.Path, ":")
-		if i < 0 {
-			return "", "", fmt.Errorf("Invalid git url %s", repoURL)
-		}
-		url.Host = url.Path[:i]
-		url.Path = "/" + url.Path[i+1:]
-	}
-
-	repoURL = "https://" + url.Host + url.Path
-
-	// add .git suffix if missing
-	if !strings.HasSuffix(repoURL, ".git") {
-		repoURL += ".git"
-	}
-
-	return repoURL, url.Host, nil
-}
-
-// sanitizeGit cleans up repository url and converts to ssh format for private
-// repositories if required.
-// Returns sanitized url, hostName (e.g. github.com, bitbucket.com)
-// and possible error
-func sanitizeGit(repoURL string) (string, string, error) {
-	repoURL = strings.TrimSpace(repoURL)
-
-	// check if valid ssh format
-	if !strings.HasPrefix(repoURL, "git@") || strings.Index(repoURL, ":") < len("git@a:") {
-		// check if valid http format and convert to ssh
-		if url, err := url.Parse(repoURL); err == nil && strings.HasPrefix(url.Scheme, "http") {
-			repoURL = fmt.Sprintf("git@%v:%v", url.Host, url.Path[1:])
-		} else {
-			return "", "", fmt.Errorf("Invalid git url %s", repoURL)
-		}
-	}
-	hostURL := repoURL[len("git@"):]
-	i := strings.Index(hostURL, ":")
-	host := hostURL[:i]
-
-	// add .git suffix if missing
-	if !strings.HasSuffix(repoURL, ".git") {
-		repoURL += ".git"
-	}
-
-	return repoURL, host, nil
-}

config/setup/git_test.go

@@ -1,144 +0,0 @@
-package setup
-
-import (
-	"testing"
-	"time"
-
-	"github.com/mholt/caddy/middleware/git"
-	"github.com/mholt/caddy/middleware/git/gittest"
-)
-
-// init sets the OS used to fakeOS
-func init() {
-	git.SetOS(gittest.FakeOS)
-}
-
-func TestGit(t *testing.T) {
-	c := newTestController(`git git@github.com:mholt/caddy.git`)
-
-	mid, err := Git(c)
-	if err != nil {
-		t.Errorf("Expected no errors, but got: %v", err)
-	}
-	if mid != nil {
-		t.Fatal("Git middleware is a background service and expected to be nil.")
-	}
-}
-
-func TestGitParse(t *testing.T) {
-	tests := []struct {
-		input     string
-		shouldErr bool
-		expected  *git.Repo
-	}{
-		{`git git@github.com:user/repo`, false, &git.Repo{
-			URL: "https://github.com/user/repo.git",
-		}},
-		{`git github.com/user/repo`, false, &git.Repo{
-			URL: "https://github.com/user/repo.git",
-		}},
-		{`git git@github.com/user/repo`, true, nil},
-		{`git http://github.com/user/repo`, false, &git.Repo{
-			URL: "https://github.com/user/repo.git",
-		}},
-		{`git https://github.com/user/repo`, false, &git.Repo{
-			URL: "https://github.com/user/repo.git",
-		}},
-		{`git http://github.com/user/repo {
-			key ~/.key
-		}`, false, &git.Repo{
-			KeyPath: "~/.key",
-			URL:     "git@github.com:user/repo.git",
-		}},
-		{`git git@github.com:user/repo {
-			key ~/.key
-		}`, false, &git.Repo{
-			KeyPath: "~/.key",
-			URL:     "git@github.com:user/repo.git",
-		}},
-		{`git `, true, nil},
-		{`git {
-		}`, true, nil},
-		{`git {
-		repo git@github.com:user/repo.git`, true, nil},
-		{`git {
-		repo git@github.com:user/repo
-		key ~/.key
-		}`, false, &git.Repo{
-			KeyPath: "~/.key",
-			URL:     "git@github.com:user/repo.git",
-		}},
-		{`git {
-		repo git@github.com:user/repo
-		key ~/.key
-		interval 600
-		}`, false, &git.Repo{
-			KeyPath:  "~/.key",
-			URL:      "git@github.com:user/repo.git",
-			Interval: time.Second * 600,
-		}},
-		{`git {
-		repo git@github.com:user/repo
-		branch dev
-		}`, false, &git.Repo{
-			Branch: "dev",
-			URL:    "https://github.com/user/repo.git",
-		}},
-		{`git {
-		key ~/.key
-		}`, true, nil},
-		{`git {
-		repo git@github.com:user/repo
-		key ~/.key
-		then echo hello world
-		}`, false, &git.Repo{
-			KeyPath: "~/.key",
-			URL:     "git@github.com:user/repo.git",
-			Then:    "echo hello world",
-		}},
-	}
-
-	for i, test := range tests {
-		c := newTestController(test.input)
-		repo, err := gitParse(c)
-		if !test.shouldErr && err != nil {
-			t.Errorf("Test %v should not error but found %v", i, err)
-			continue
-		}
-		if test.shouldErr && err == nil {
-			t.Errorf("Test %v should error but found nil", i)
-			continue
-		}
-		if !reposEqual(test.expected, repo) {
-			t.Errorf("Test %v expects %v but found %v", i, test.expected, repo)
-		}
-	}
-}
-
-func reposEqual(expected, repo *git.Repo) bool {
-	if expected == nil {
-		return repo == nil
-	}
-	if expected.Branch != "" && expected.Branch != repo.Branch {
-		return false
-	}
-	if expected.Host != "" && expected.Host != repo.Host {
-		return false
-	}
-	if expected.Interval != 0 && expected.Interval != repo.Interval {
-		return false
-	}
-	if expected.KeyPath != "" && expected.KeyPath != repo.KeyPath {
-		return false
-	}
-	if expected.Path != "" && expected.Path != repo.Path {
-		return false
-	}
-	if expected.Then != "" && expected.Then != repo.Then {
-		return false
-	}
-	if expected.URL != "" && expected.URL != repo.URL {
-		return false
-	}
-	return true
-}

config/setup/gzip.go

@@ -1,13 +0,0 @@
-package setup
-
-import (
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/gzip"
-)
-
-// Gzip configures a new gzip middleware instance.
-func Gzip(c *Controller) (middleware.Middleware, error) {
-	return func(next middleware.Handler) middleware.Handler {
-		return gzip.Gzip{Next: next}
-	}, nil
-}

config/setup/gzip_test.go

@@ -1,29 +0,0 @@
-package setup
-
-import (
-	"testing"
-
-	"github.com/mholt/caddy/middleware/gzip"
-)
-
-func TestGzip(t *testing.T) {
-	c := newTestController(`gzip`)
-
-	mid, err := Gzip(c)
-	if err != nil {
-		t.Errorf("Expected no errors, but got: %v", err)
-	}
-	if mid == nil {
-		t.Fatal("Expected middleware, was nil instead")
-	}
-
-	handler := mid(emptyNext)
-	myHandler, ok := handler.(gzip.Gzip)
-	if !ok {
-		t.Fatalf("Expected handler to be type Gzip, got: %#v", handler)
-	}
-
-	if !sameNext(myHandler.Next, emptyNext) {
-		t.Error("'Next' field of handler was not set properly")
-	}
-}

config/setup/headers.go

@@ -1,84 +0,0 @@
-package setup
-
-import (
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/headers"
-)
-
-// Headers configures a new Headers middleware instance.
-func Headers(c *Controller) (middleware.Middleware, error) {
-	rules, err := headersParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	return func(next middleware.Handler) middleware.Handler {
-		return headers.Headers{Next: next, Rules: rules}
-	}, nil
-}
-
-func headersParse(c *Controller) ([]headers.Rule, error) {
-	var rules []headers.Rule
-
-	for c.NextLine() {
-		var head headers.Rule
-		var isNewPattern bool
-
-		if !c.NextArg() {
-			return rules, c.ArgErr()
-		}
-		pattern := c.Val()
-
-		// See if we already have a definition for this Path pattern...
-		for _, h := range rules {
-			if h.Path == pattern {
-				head = h
-				break
-			}
-		}
-
-		// ...otherwise, this is a new pattern
-		if head.Path == "" {
-			head.Path = pattern
-			isNewPattern = true
-		}
-
-		for c.NextBlock() {
-			// A block of headers was opened...
-
-			h := headers.Header{Name: c.Val()}
-
-			if c.NextArg() {
-				h.Value = c.Val()
-			}
-
-			head.Headers = append(head.Headers, h)
-		}
-		if c.NextArg() {
-			// ... or single header was defined as an argument instead.
-
-			h := headers.Header{Name: c.Val()}
-
-			h.Value = c.Val()
-
-			if c.NextArg() {
-				h.Value = c.Val()
-			}
-
-			head.Headers = append(head.Headers, h)
-		}
-
-		if isNewPattern {
-			rules = append(rules, head)
-		} else {
-			for i := 0; i < len(rules); i++ {
-				if rules[i].Path == pattern {
-					rules[i] = head
-					break
-				}
-			}
-		}
-	}
-
-	return rules, nil
-}

config/setup/headers_test.go

@@ -1,85 +0,0 @@
-package setup
-
-import (
-	"fmt"
-	"testing"
-
-	"github.com/mholt/caddy/middleware/headers"
-)
-
-func TestHeaders(t *testing.T) {
-	c := newTestController(`header / Foo Bar`)
-
-	mid, err := Headers(c)
-	if err != nil {
-		t.Errorf("Expected no errors, but got: %v", err)
-	}
-	if mid == nil {
-		t.Fatal("Expected middleware, was nil instead")
-	}
-
-	handler := mid(emptyNext)
-	myHandler, ok := handler.(headers.Headers)
-	if !ok {
-		t.Fatalf("Expected handler to be type Headers, got: %#v", handler)
-	}
-
-	if !sameNext(myHandler.Next, emptyNext) {
-		t.Error("'Next' field of handler was not set properly")
-	}
-}
-
-func TestHeadersParse(t *testing.T) {
-	tests := []struct {
-		input     string
-		shouldErr bool
-		expected  []headers.Rule
-	}{
-		{`header /foo Foo "Bar Baz"`,
-			false, []headers.Rule{
-				{Path: "/foo", Headers: []headers.Header{
-					{"Foo", "Bar Baz"},
-				}},
-			}},
-		{`header /bar { Foo "Bar Baz" Baz Qux }`,
-			false, []headers.Rule{
-				{Path: "/bar", Headers: []headers.Header{
-					{"Foo", "Bar Baz"},
-					{"Baz", "Qux"},
-				}},
-			}},
-	}
-
-	for i, test := range tests {
-		c := newTestController(test.input)
-		actual, err := headersParse(c)
-
-		if err == nil && test.shouldErr {
-			t.Errorf("Test %d didn't error, but it should have", i)
-		} else if err != nil && !test.shouldErr {
-			t.Errorf("Test %d errored, but it shouldn't have; got '%v'", i, err)
-		}
-
-		if len(actual) != len(test.expected) {
-			t.Fatalf("Test %d expected %d rules, but got %d",
-				i, len(test.expected), len(actual))
-		}
-
-		for j, expectedRule := range test.expected {
-			actualRule := actual[j]
-
-			if actualRule.Path != expectedRule.Path {
-				t.Errorf("Test %d, rule %d: Expected path %s, but got %s",
-					i, j, expectedRule.Path, actualRule.Path)
-			}
-
-			expectedHeaders := fmt.Sprintf("%v", expectedRule.Headers)
-			actualHeaders := fmt.Sprintf("%v", actualRule.Headers)
-
-			if actualHeaders != expectedHeaders {
-				t.Errorf("Test %d, rule %d: Expected headers %s, but got %s",
-					i, j, expectedHeaders, actualHeaders)
-			}
-		}
-	}
-}

config/setup/internal.go

@@ -1,31 +0,0 @@
-package setup
-
-import (
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/internal"
-)
-
-// Internal configures a new Internal middleware instance.
-func Internal(c *Controller) (middleware.Middleware, error) {
-	paths, err := internalParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	return func(next middleware.Handler) middleware.Handler {
-		return internal.Internal{Next: next, Paths: paths}
-	}, nil
-}
-
-func internalParse(c *Controller) ([]string, error) {
-	var paths []string
-
-	for c.Next() {
-		if !c.NextArg() {
-			return paths, c.ArgErr()
-		}
-		paths = append(paths, c.Val())
-	}
-
-	return paths, nil
-}

config/setup/log.go

@@ -1,91 +0,0 @@
-package setup
-
-import (
-	"log"
-	"os"
-
-	"github.com/mholt/caddy/middleware"
-	caddylog "github.com/mholt/caddy/middleware/log"
-)
-
-// Log sets up the logging middleware.
-func Log(c *Controller) (middleware.Middleware, error) {
-	rules, err := logParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	// Open the log files for writing when the server starts
-	c.Startup = append(c.Startup, func() error {
-		for i := 0; i < len(rules); i++ {
-			var err error
-			var file *os.File
-
-			if rules[i].OutputFile == "stdout" {
-				file = os.Stdout
-			} else if rules[i].OutputFile == "stderr" {
-				file = os.Stderr
-			} else {
-				file, err = os.OpenFile(rules[i].OutputFile, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0644)
-				if err != nil {
-					return err
-				}
-			}
-
-			rules[i].Log = log.New(file, "", 0)
-		}
-
-		return nil
-	})
-
-	return func(next middleware.Handler) middleware.Handler {
-		return caddylog.Logger{Next: next, Rules: rules}
-	}, nil
-}
-
-func logParse(c *Controller) ([]caddylog.Rule, error) {
-	var rules []caddylog.Rule
-
-	for c.Next() {
-		args := c.RemainingArgs()
-
-		if len(args) == 0 {
-			// Nothing specified; use defaults
-			rules = append(rules, caddylog.Rule{
-				PathScope:  "/",
-				OutputFile: caddylog.DefaultLogFilename,
-				Format:     caddylog.DefaultLogFormat,
-			})
-		} else if len(args) == 1 {
-			// Only an output file specified
-			rules = append(rules, caddylog.Rule{
-				PathScope:  "/",
-				OutputFile: args[0],
-				Format:     caddylog.DefaultLogFormat,
-			})
-		} else {
-			// Path scope, output file, and maybe a format specified
-
-			format := caddylog.DefaultLogFormat
-
-			if len(args) > 2 {
-				switch args[2] {
-				case "{common}":
-					format = caddylog.CommonLogFormat
-				case "{combined}":
-					format = caddylog.CombinedLogFormat
-				default:
-					format = args[2]
-				}
-			}
-
-			rules = append(rules, caddylog.Rule{
-				PathScope:  args[0],
-				OutputFile: args[1],
-				Format:     format,
-			})
-		}
-	}
-
-	return rules, nil
-}

config/setup/markdown.go

@@ -1,163 +0,0 @@
-package setup
-
-import (
-	"io/ioutil"
-	"net/http"
-	"os"
-	"path"
-	"path/filepath"
-	"strings"
-
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/markdown"
-	"github.com/russross/blackfriday"
-)
-
-// Markdown configures a new Markdown middleware instance.
-func Markdown(c *Controller) (middleware.Middleware, error) {
-	mdconfigs, err := markdownParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	md := markdown.Markdown{
-		Root:       c.Root,
-		FileSys:    http.Dir(c.Root),
-		Configs:    mdconfigs,
-		IndexFiles: []string{"index.md"},
-	}
-
-	// For any configs that enabled static site gen, sweep the whole path at startup
-	c.Startup = append(c.Startup, func() error {
-		for _, cfg := range mdconfigs {
-			if cfg.StaticDir == "" {
-				continue
-			}
-
-			// If generated site already exists, clear it out
-			_, err := os.Stat(cfg.StaticDir)
-			if err == nil {
-				err := os.RemoveAll(cfg.StaticDir)
-				if err != nil {
-					return err
-				}
-			}
-
-			fp := filepath.Join(md.Root, cfg.PathScope)
-			filepath.Walk(fp, func(path string, info os.FileInfo, err error) error {
-				for _, ext := range cfg.Extensions {
-					if !info.IsDir() && strings.HasSuffix(info.Name(), ext) {
-						// Load the file
-						body, err := ioutil.ReadFile(path)
-						if err != nil {
-							return err
-						}
-
-						// Get the relative path as if it were a HTTP request,
-						// then prepend with "/" (like a real HTTP request)
-						reqPath, err := filepath.Rel(md.Root, path)
-						if err != nil {
-							return err
-						}
-						reqPath = "/" + reqPath
-
-						// Generate the static file
-						_, err = md.Process(cfg, reqPath, body)
-						if err != nil {
-							return err
-						}
-
-						break // don't try other file extensions
-					}
-				}
-
-				return nil
-			})
-		}
-
-		return nil
-	})
-
-	return func(next middleware.Handler) middleware.Handler {
-		md.Next = next
-		return md
-	}, nil
-}
-
-func markdownParse(c *Controller) ([]markdown.Config, error) {
-	var mdconfigs []markdown.Config
-
-	for c.Next() {
-		md := markdown.Config{
-			Renderer:    blackfriday.HtmlRenderer(0, "", ""),
-			Templates:   make(map[string]string),
-			StaticFiles: make(map[string]string),
-		}
-
-		// Get the path scope
-		if !c.NextArg() || c.Val() == "{" {
-			return mdconfigs, c.ArgErr()
-		}
-		md.PathScope = c.Val()
-
-		// Load any other configuration parameters
-		for c.NextBlock() {
-			switch c.Val() {
-			case "ext":
-				exts := c.RemainingArgs()
-				if len(exts) == 0 {
-					return mdconfigs, c.ArgErr()
-				}
-				md.Extensions = append(md.Extensions, exts...)
-			case "css":
-				if !c.NextArg() {
-					return mdconfigs, c.ArgErr()
-				}
-				md.Styles = append(md.Styles, c.Val())
-			case "js":
-				if !c.NextArg() {
-					return mdconfigs, c.ArgErr()
-				}
-				md.Scripts = append(md.Scripts, c.Val())
-			case "template":
-				tArgs := c.RemainingArgs()
-				switch len(tArgs) {
-				case 0:
-					return mdconfigs, c.ArgErr()
-				case 1:
-					if _, ok := md.Templates[markdown.DefaultTemplate]; ok {
-						return mdconfigs, c.Err("only one default template is allowed, use alias.")
-					}
-					fpath := filepath.Clean(c.Root + string(filepath.Separator) + tArgs[0])
-					md.Templates[markdown.DefaultTemplate] = fpath
-				case 2:
-					fpath := filepath.Clean(c.Root + string(filepath.Separator) + tArgs[1])
-					md.Templates[tArgs[0]] = fpath
-				default:
-					return mdconfigs, c.ArgErr()
-				}
-			case "sitegen":
-				if c.NextArg() {
-					md.StaticDir = path.Join(c.Root, c.Val())
-				} else {
-					md.StaticDir = path.Join(c.Root, markdown.DefaultStaticDir)
-				}
-				if c.NextArg() {
-					// only 1 argument allowed
-					return mdconfigs, c.ArgErr()
-				}
-			default:
-				return mdconfigs, c.Err("Expected valid markdown configuration property")
-			}
-		}
-
-		// If no extensions were specified, assume .md
-		if len(md.Extensions) == 0 {
-			md.Extensions = []string{".md"}
-		}
-
-		mdconfigs = append(mdconfigs, md)
-	}
-
-	return mdconfigs, nil
-}

config/setup/proxy.go

@@ -1,17 +0,0 @@
-package setup
-
-import (
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/proxy"
-)
-
-// Proxy configures a new Proxy middleware instance.
-func Proxy(c *Controller) (middleware.Middleware, error) {
-	if upstreams, err := proxy.NewStaticUpstreams(c.Dispenser); err == nil {
-		return func(next middleware.Handler) middleware.Handler {
-			return proxy.Proxy{Next: next, Upstreams: upstreams}
-		}, nil
-	} else {
-		return nil, err
-	}
-}

config/setup/redir.go

@@ -1,77 +0,0 @@
-package setup
-
-import (
-	"net/http"
-
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/redirect"
-)
-
-// Redir configures a new Redirect middleware instance.
-func Redir(c *Controller) (middleware.Middleware, error) {
-	rules, err := redirParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	return func(next middleware.Handler) middleware.Handler {
-		return redirect.Redirect{Next: next, Rules: rules}
-	}, nil
-}
-
-func redirParse(c *Controller) ([]redirect.Rule, error) {
-	var redirects []redirect.Rule
-
-	for c.Next() {
-		var rule redirect.Rule
-		args := c.RemainingArgs()
-
-		switch len(args) {
-		case 1:
-			// To specified
-			rule.From = "/"
-			rule.To = args[0]
-			rule.Code = http.StatusMovedPermanently
-		case 2:
-			// To and Code specified
-			rule.From = "/"
-			rule.To = args[0]
-			if code, ok := httpRedirs[args[1]]; !ok {
-				return redirects, c.Err("Invalid redirect code '" + args[1] + "'")
-			} else {
-				rule.Code = code
-			}
-		case 3:
-			// From, To, and Code specified
-			rule.From = args[0]
-			rule.To = args[1]
-			if code, ok := httpRedirs[args[2]]; !ok {
-				return redirects, c.Err("Invalid redirect code '" + args[2] + "'")
-			} else {
-				rule.Code = code
-			}
-		default:
-			return redirects, c.ArgErr()
-		}
-
-		if rule.From == rule.To {
-			return redirects, c.Err("Redirect rule cannot allow From and To arguments to be the same.")
-		}
-
-		redirects = append(redirects, rule)
-	}
-
-	return redirects, nil
-}
-
-// httpRedirs is a list of supported HTTP redirect codes.
-var httpRedirs = map[string]int{
-	"300": 300,
-	"301": 301,
-	"302": 302,
-	"303": 303,
-	"304": 304,
-	"305": 305,
-	"307": 307,
-	"308": 308,
-}

config/setup/rewrite.go

@@ -1,79 +0,0 @@
-package setup
-
-import (
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/rewrite"
-)
-
-// Rewrite configures a new Rewrite middleware instance.
-func Rewrite(c *Controller) (middleware.Middleware, error) {
-	rewrites, err := rewriteParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	return func(next middleware.Handler) middleware.Handler {
-		return rewrite.Rewrite{Next: next, Rules: rewrites}
-	}, nil
-}
-
-func rewriteParse(c *Controller) ([]rewrite.Rule, error) {
-	var simpleRules []rewrite.Rule
-	var regexpRules []rewrite.Rule
-
-	for c.Next() {
-		var rule rewrite.Rule
-		var err error
-		var base = "/"
-		var pattern, to string
-		var ext []string
-
-		args := c.RemainingArgs()
-
-		switch len(args) {
-		case 2:
-			rule = rewrite.NewSimpleRule(args[0], args[1])
-			simpleRules = append(simpleRules, rule)
-		case 1:
-			base = args[0]
-			fallthrough
-		case 0:
-			for c.NextBlock() {
-				switch c.Val() {
-				case "r", "regexp":
-					if !c.NextArg() {
-						return nil, c.ArgErr()
-					}
-					pattern = c.Val()
-				case "to":
-					if !c.NextArg() {
-						return nil, c.ArgErr()
-					}
-					to = c.Val()
-				case "ext":
-					args1 := c.RemainingArgs()
-					if len(args1) == 0 {
-						return nil, c.ArgErr()
-					}
-					ext = args1
-				default:
-					return nil, c.ArgErr()
-				}
-			}
-			// ensure pattern and to are specified
-			if pattern == "" || to == "" {
-				return nil, c.ArgErr()
-			}
-			if rule, err = rewrite.NewRegexpRule(base, pattern, to, ext); err != nil {
-				return nil, err
-			}
-			regexpRules = append(regexpRules, rule)
-		default:
-			return nil, c.ArgErr()
-		}
-
-	}
-
-	// put simple rules in front to avoid regexp computation for them
-	return append(simpleRules, regexpRules...), nil
-}

config/setup/rewrite_test.go

@@ -1,184 +0,0 @@
-package setup
-
-import (
-	"testing"
-
-	"fmt"
-	"github.com/mholt/caddy/middleware/rewrite"
-	"regexp"
-)
-
-func TestRewrite(t *testing.T) {
-	c := newTestController(`rewrite /from /to`)
-
-	mid, err := Rewrite(c)
-	if err != nil {
-		t.Errorf("Expected no errors, but got: %v", err)
-	}
-	if mid == nil {
-		t.Fatal("Expected middleware, was nil instead")
-	}
-
-	handler := mid(emptyNext)
-	myHandler, ok := handler.(rewrite.Rewrite)
-	if !ok {
-		t.Fatalf("Expected handler to be type Rewrite, got: %#v", handler)
-	}
-
-	if !sameNext(myHandler.Next, emptyNext) {
-		t.Error("'Next' field of handler was not set properly")
-	}
-
-	if len(myHandler.Rules) != 1 {
-		t.Errorf("Expected handler to have %d rule, has %d instead", 1, len(myHandler.Rules))
-	}
-}
-
-func TestRewriteParse(t *testing.T) {
-	simpleTests := []struct {
-		input     string
-		shouldErr bool
-		expected  []rewrite.Rule
-	}{
-		{`rewrite /from /to`, false, []rewrite.Rule{
-			rewrite.SimpleRule{"/from", "/to"},
-		}},
-		{`rewrite /from /to
-		  rewrite a b`, false, []rewrite.Rule{
-			rewrite.SimpleRule{"/from", "/to"},
-			rewrite.SimpleRule{"a", "b"},
-		}},
-		{`rewrite a`, true, []rewrite.Rule{}},
-		{`rewrite`, true, []rewrite.Rule{}},
-		{`rewrite a b c`, true, []rewrite.Rule{
-			rewrite.SimpleRule{"a", "b"},
-		}},
-	}
-
-	for i, test := range simpleTests {
-		c := newTestController(test.input)
-		actual, err := rewriteParse(c)
-
-		if err == nil && test.shouldErr {
-			t.Errorf("Test %d didn't error, but it should have", i)
-		} else if err != nil && !test.shouldErr {
-			t.Errorf("Test %d errored, but it shouldn't have; got '%v'", i, err)
-		} else if err != nil && test.shouldErr {
-			continue
-		}
-
-		if len(actual) != len(test.expected) {
-			t.Fatalf("Test %d expected %d rules, but got %d",
-				i, len(test.expected), len(actual))
-		}
-
-		for j, e := range test.expected {
-			actualRule := actual[j].(rewrite.SimpleRule)
-			expectedRule := e.(rewrite.SimpleRule)
-
-			if actualRule.From != expectedRule.From {
-				t.Errorf("Test %d, rule %d: Expected From=%s, got %s",
-					i, j, expectedRule.From, actualRule.From)
-			}
-
-			if actualRule.To != expectedRule.To {
-				t.Errorf("Test %d, rule %d: Expected To=%s, got %s",
-					i, j, expectedRule.To, actualRule.To)
-			}
-		}
-	}
-
-	regexpTests := []struct {
-		input     string
-		shouldErr bool
-		expected  []rewrite.Rule
-	}{
-		{`rewrite {
-			r	.*
-			to	/to
-		 }`, false, []rewrite.Rule{
-			&rewrite.RegexpRule{"/", "/to", nil, regexp.MustCompile(".*")},
-		}},
-		{`rewrite {
-			regexp	.*
-			to		/to
-			ext		/ html txt
-		 }`, false, []rewrite.Rule{
-			&rewrite.RegexpRule{"/", "/to", []string{"/", "html", "txt"}, regexp.MustCompile(".*")},
-		}},
-		{`rewrite /path {
-			r	rr
-			to	/dest
-		 }
-		 rewrite / {
-		 	regexp	[a-z]+
-		 	to 		/to
-		 }
-		 `, false, []rewrite.Rule{
-			&rewrite.RegexpRule{"/path", "/dest", nil, regexp.MustCompile("rr")},
-			&rewrite.RegexpRule{"/", "/to", nil, regexp.MustCompile("[a-z]+")},
-		}},
-		{`rewrite {
-			to	/to
-		 }`, true, []rewrite.Rule{
-			&rewrite.RegexpRule{},
-		}},
-		{`rewrite {
-			r	.*
-		 }`, true, []rewrite.Rule{
-			&rewrite.RegexpRule{},
-		}},
-		{`rewrite {
-
-		 }`, true, []rewrite.Rule{
-			&rewrite.RegexpRule{},
-		}},
-		{`rewrite /`, true, []rewrite.Rule{
-			&rewrite.RegexpRule{},
-		}},
-	}
-
-	for i, test := range regexpTests {
-		c := newTestController(test.input)
-		actual, err := rewriteParse(c)
-
-		if err == nil && test.shouldErr {
-			t.Errorf("Test %d didn't error, but it should have", i)
-		} else if err != nil && !test.shouldErr {
-			t.Errorf("Test %d errored, but it shouldn't have; got '%v'", i, err)
-		} else if err != nil && test.shouldErr {
-			continue
-		}
-
-		if len(actual) != len(test.expected) {
-			t.Fatalf("Test %d expected %d rules, but got %d",
-				i, len(test.expected), len(actual))
-		}
-
-		for j, e := range test.expected {
-			actualRule := actual[j].(*rewrite.RegexpRule)
-			expectedRule := e.(*rewrite.RegexpRule)
-
-			if actualRule.Base != expectedRule.Base {
-				t.Errorf("Test %d, rule %d: Expected Base=%s, got %s",
-					i, j, expectedRule.Base, actualRule.Base)
-			}
-
-			if actualRule.To != expectedRule.To {
-				t.Errorf("Test %d, rule %d: Expected To=%s, got %s",
-					i, j, expectedRule.To, actualRule.To)
-			}
-
-			if fmt.Sprint(actualRule.Exts) != fmt.Sprint(expectedRule.Exts) {
-				t.Errorf("Test %d, rule %d: Expected Ext=%v, got %v",
-					i, j, expectedRule.To, actualRule.To)
-			}
-
-			if actualRule.String() != expectedRule.String() {
-				t.Errorf("Test %d, rule %d: Expected Pattern=%s, got %s",
-					i, j, expectedRule.String(), actualRule.String())
-			}
-		}
-	}
-
-}

config/setup/root.go

@@ -1,31 +0,0 @@
-package setup
-
-import (
-	"log"
-	"os"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-func Root(c *Controller) (middleware.Middleware, error) {
-	for c.Next() {
-		if !c.NextArg() {
-			return nil, c.ArgErr()
-		}
-		c.Root = c.Val()
-	}
-
-	// Check if root path exists
-	_, err := os.Stat(c.Root)
-	if err != nil {
-		if os.IsNotExist(err) {
-			// Allow this, because the folder might appear later.
-			// But make sure the user knows!
-			log.Printf("Warning: Root path does not exist: %s", c.Root)
-		} else {
-			return nil, c.Errf("Unable to access root path '%s': %v", c.Root, err)
-		}
-	}
-
-	return nil, nil
-}

config/setup/startupshutdown.go

@@ -1,58 +0,0 @@
-package setup
-
-import (
-	"os"
-	"os/exec"
-	"strings"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-func Startup(c *Controller) (middleware.Middleware, error) {
-	return nil, registerCallback(c, &c.Startup)
-}
-
-func Shutdown(c *Controller) (middleware.Middleware, error) {
-	return nil, registerCallback(c, &c.Shutdown)
-}
-
-// registerCallback registers a callback function to execute by
-// using c to parse the line. It appends the callback function
-// to the list of callback functions passed in by reference.
-func registerCallback(c *Controller, list *[]func() error) error {
-	for c.Next() {
-		args := c.RemainingArgs()
-		if len(args) == 0 {
-			return c.ArgErr()
-		}
-
-		nonblock := false
-		if len(args) > 1 && args[len(args)-1] == "&" {
-			// Run command in background; non-blocking
-			nonblock = true
-			args = args[:len(args)-1]
-		}
-
-		command, args, err := middleware.SplitCommandAndArgs(strings.Join(args, " "))
-		if err != nil {
-			return c.Err(err.Error())
-		}
-
-		fn := func() error {
-			cmd := exec.Command(command, args...)
-			cmd.Stdin = os.Stdin
-			cmd.Stdout = os.Stdout
-			cmd.Stderr = os.Stderr
-
-			if nonblock {
-				return cmd.Start()
-			} else {
-				return cmd.Run()
-			}
-		}
-
-		*list = append(*list, fn)
-	}
-
-	return nil
-}

config/setup/templates.go

@@ -1,61 +0,0 @@
-package setup
-
-import (
-	"net/http"
-
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/templates"
-)
-
-// Templates configures a new Templates middleware instance.
-func Templates(c *Controller) (middleware.Middleware, error) {
-	rules, err := templatesParse(c)
-	if err != nil {
-		return nil, err
-	}
-
-	tmpls := templates.Templates{
-		Rules:   rules,
-		Root:    c.Root,
-		FileSys: http.Dir(c.Root),
-	}
-
-	return func(next middleware.Handler) middleware.Handler {
-		tmpls.Next = next
-		return tmpls
-	}, nil
-}
-
-func templatesParse(c *Controller) ([]templates.Rule, error) {
-	var rules []templates.Rule
-
-	for c.Next() {
-		var rule templates.Rule
-
-		if c.NextArg() {
-			// First argument would be the path
-			rule.Path = c.Val()
-
-			// Any remaining arguments are extensions
-			rule.Extensions = c.RemainingArgs()
-			if len(rule.Extensions) == 0 {
-				rule.Extensions = defaultExtensions
-			}
-		} else {
-			rule.Path = defaultPath
-			rule.Extensions = defaultExtensions
-		}
-
-		for _, ext := range rule.Extensions {
-			rule.IndexFiles = append(rule.IndexFiles, "index"+ext)
-		}
-
-		rules = append(rules, rule)
-	}
-
-	return rules, nil
-}
-
-const defaultPath = "/"
-
-var defaultExtensions = []string{".html", ".htm", ".tmpl", ".tpl", ".txt"}

config/setup/tls.go

@@ -1,134 +0,0 @@
-package setup
-
-import (
-	"crypto/tls"
-	"log"
-	"strings"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-func TLS(c *Controller) (middleware.Middleware, error) {
-	c.TLS.Enabled = true
-	if c.Port == "http" {
-		c.TLS.Enabled = false
-		log.Printf("Warning: TLS disabled for %s://%s. To force TLS over the plaintext HTTP port, "+
-			"specify port 80 explicitly (https://%s:80).", c.Port, c.Host, c.Host)
-	}
-
-	for c.Next() {
-		if !c.NextArg() {
-			return nil, c.ArgErr()
-		}
-		c.TLS.Certificate = c.Val()
-
-		if !c.NextArg() {
-			return nil, c.ArgErr()
-		}
-		c.TLS.Key = c.Val()
-
-		// Optional block
-		for c.NextBlock() {
-			switch c.Val() {
-			case "protocols":
-				args := c.RemainingArgs()
-				if len(args) != 2 {
-					return nil, c.ArgErr()
-				}
-				value, ok := supportedProtocols[strings.ToLower(args[0])]
-				if !ok {
-					return nil, c.Errf("Wrong protocol name or protocol not supported '%s'", c.Val())
-				}
-				c.TLS.ProtocolMinVersion = value
-				value, ok = supportedProtocols[strings.ToLower(args[1])]
-				if !ok {
-					return nil, c.Errf("Wrong protocol name or protocol not supported '%s'", c.Val())
-				}
-				c.TLS.ProtocolMaxVersion = value
-			case "ciphers":
-				for c.NextArg() {
-					value, ok := supportedCiphersMap[strings.ToUpper(c.Val())]
-					if !ok {
-						return nil, c.Errf("Wrong cipher name or cipher not supported '%s'", c.Val())
-					}
-					c.TLS.Ciphers = append(c.TLS.Ciphers, value)
-				}
-			default:
-				return nil, c.Errf("Unknown keyword '%s'")
-			}
-		}
-	}
-
-	// If no ciphers provided, use all that Caddy supports for the protocol
-	if len(c.TLS.Ciphers) == 0 {
-		c.TLS.Ciphers = supportedCiphers
-	}
-
-	// Not a cipher suite, but still important for mitigating protocol downgrade attacks
-	c.TLS.Ciphers = append(c.TLS.Ciphers, tls.TLS_FALLBACK_SCSV)
-
-	// Set default protocol min and max versions - must balance compatibility and security
-	if c.TLS.ProtocolMinVersion == 0 {
-		c.TLS.ProtocolMinVersion = tls.VersionTLS10
-	}
-	if c.TLS.ProtocolMaxVersion == 0 {
-		c.TLS.ProtocolMaxVersion = tls.VersionTLS12
-	}
-
-	// Prefer server cipher suites
-	c.TLS.PreferServerCipherSuites = true
-
-	return nil, nil
-}
-
-// Map of supported protocols
-// SSLv3 will be not supported in future release
-// HTTP/2 only supports TLS 1.2 and higher
-var supportedProtocols = map[string]uint16{
-	"ssl3.0": tls.VersionSSL30,
-	"tls1.0": tls.VersionTLS10,
-	"tls1.1": tls.VersionTLS11,
-	"tls1.2": tls.VersionTLS12,
-}
-
-// Map of supported ciphers, used only for parsing config.
-//
-// Note that, at time of writing, HTTP/2 blacklists 276 cipher suites,
-// including all but two of the suites below (the two GCM suites).
-// See https://http2.github.io/http2-spec/#BadCipherSuites
-//
-// TLS_FALLBACK_SCSV is not in this list because we manually ensure
-// it is always added (even though it is not technically a cipher suite).
-//
-// This map, like any map, is NOT ORDERED. Do not range over this map.
-var supportedCiphersMap = map[string]uint16{
-	"ECDHE-RSA-AES128-GCM-SHA256":   tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
-	"ECDHE-ECDSA-AES128-GCM-SHA256": tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
-	"ECDHE-RSA-AES128-CBC-SHA":      tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
-	"ECDHE-RSA-AES256-CBC-SHA":      tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
-	"ECDHE-ECDSA-AES256-CBC-SHA":    tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
-	"ECDHE-ECDSA-AES128-CBC-SHA":    tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
-	"RSA-AES128-CBC-SHA":            tls.TLS_RSA_WITH_AES_128_CBC_SHA,
-	"RSA-AES256-CBC-SHA":            tls.TLS_RSA_WITH_AES_256_CBC_SHA,
-	"ECDHE-RSA-3DES-EDE-CBC-SHA":    tls.TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,
-	"RSA-3DES-EDE-CBC-SHA":          tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,
-}
-
-// List of supported cipher suites in descending order of preference.
-// Ordering is very important! Getting the wrong order will break
-// mainstream clients, especially with HTTP/2.
-//
-// Note that TLS_FALLBACK_SCSV is not in this list since it is always
-// added manually.
-var supportedCiphers = []uint16{
-	tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
-	tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
-	tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
-	tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
-	tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
-	tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
-	tls.TLS_RSA_WITH_AES_256_CBC_SHA,
-	tls.TLS_RSA_WITH_AES_128_CBC_SHA,
-	tls.TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,
-	tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,
-}

config/setup/tls_test.go

@@ -1,129 +0,0 @@
-package setup
-
-import (
-	"crypto/tls"
-	"testing"
-)
-
-func TestTLSParseBasic(t *testing.T) {
-	c := newTestController(`tls cert.pem key.pem`)
-
-	_, err := TLS(c)
-	if err != nil {
-		t.Errorf("Expected no errors, got: %v", err)
-	}
-
-	// Basic checks
-	if c.TLS.Certificate != "cert.pem" {
-		t.Errorf("Expected certificate arg to be 'cert.pem', was '%s'", c.TLS.Certificate)
-	}
-	if c.TLS.Key != "key.pem" {
-		t.Errorf("Expected key arg to be 'key.pem', was '%s'", c.TLS.Key)
-	}
-	if !c.TLS.Enabled {
-		t.Error("Expected TLS Enabled=true, but was false")
-	}
-
-	// Security defaults
-	if c.TLS.ProtocolMinVersion != tls.VersionTLS10 {
-		t.Errorf("Expected 'tls1.0 (0x0301)' as ProtocolMinVersion, got %#v", c.TLS.ProtocolMinVersion)
-	}
-	if c.TLS.ProtocolMaxVersion != tls.VersionTLS12 {
-		t.Errorf("Expected 'tls1.2 (0x0303)' as ProtocolMaxVersion, got %v", c.TLS.ProtocolMaxVersion)
-	}
-
-	// Cipher checks
-	expectedCiphers := []uint16{
-		tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
-		tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,
-		tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
-		tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,
-		tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
-		tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
-		tls.TLS_RSA_WITH_AES_256_CBC_SHA,
-		tls.TLS_RSA_WITH_AES_128_CBC_SHA,
-		tls.TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,
-		tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA,
-		tls.TLS_FALLBACK_SCSV,
-	}
-
-	// Ensure count is correct (plus one for TLS_FALLBACK_SCSV)
-	if len(c.TLS.Ciphers) != len(supportedCiphers)+1 {
-		t.Errorf("Expected %v Ciphers (including TLS_FALLBACK_SCSV), got %v",
-			len(supportedCiphers)+1, len(c.TLS.Ciphers))
-	}
-
-	// Ensure ordering is correct
-	for i, actual := range c.TLS.Ciphers {
-		if actual != expectedCiphers[i] {
-			t.Errorf("Expected cipher in position %d to be %0x, got %0x", i, expectedCiphers[i], actual)
-		}
-	}
-
-	if !c.TLS.PreferServerCipherSuites {
-		t.Error("Expected PreferServerCipherSuites = true, but was false")
-	}
-}
-
-func TestTLSParseIncompleteParams(t *testing.T) {
-	c := newTestController(`tls`)
-
-	_, err := TLS(c)
-	if err == nil {
-		t.Errorf("Expected errors, but no error returned")
-	}
-
-	c = newTestController(`tls cert.key`)
-
-	_, err = TLS(c)
-	if err == nil {
-		t.Errorf("Expected errors, but no error returned")
-	}
-}
-
-func TestTLSParseWithOptionalParams(t *testing.T) {
-	params := `tls cert.crt cert.key {
-            protocols ssl3.0 tls1.2
-            ciphers RSA-3DES-EDE-CBC-SHA RSA-AES256-CBC-SHA ECDHE-RSA-AES128-GCM-SHA256
-        }`
-	c := newTestController(params)
-
-	_, err := TLS(c)
-	if err != nil {
-		t.Errorf("Expected no errors, got: %v", err)
-	}
-
-	if c.TLS.ProtocolMinVersion != tls.VersionSSL30 {
-		t.Errorf("Expected 'ssl3.0 (0x0300)' as ProtocolMinVersion, got %#v", c.TLS.ProtocolMinVersion)
-	}
-
-	if c.TLS.ProtocolMaxVersion != tls.VersionTLS12 {
-		t.Errorf("Expected 'tls1.2 (0x0302)' as ProtocolMaxVersion, got %#v", c.TLS.ProtocolMaxVersion)
-	}
-
-	if len(c.TLS.Ciphers)-1 != 3 {
-		t.Errorf("Expected 3 Ciphers (not including TLS_FALLBACK_SCSV), got %v", len(c.TLS.Ciphers))
-	}
-}
-
-func TestTLSParseWithWrongOptionalParams(t *testing.T) {
-	// Test protocols wrong params
-	params := `tls cert.crt cert.key {
-			protocols ssl tls
-		}`
-	c := newTestController(params)
-	_, err := TLS(c)
-	if err == nil {
-		t.Errorf("Expected errors, but no error returned")
-	}
-
-	// Test ciphers wrong params
-	params = `tls cert.crt cert.key {
-			ciphers not-valid-cipher
-		}`
-	c = newTestController(params)
-	_, err = TLS(c)
-	if err == nil {
-		t.Errorf("Expected errors, but no error returned")
-	}
-}

config/setup/websocket.go

@@ -1,77 +0,0 @@
-package setup
-
-import (
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/websockets"
-)
-
-// WebSocket configures a new WebSockets middleware instance.
-func WebSocket(c *Controller) (middleware.Middleware, error) {
-	var websocks []websockets.Config
-	var respawn bool
-
-	optionalBlock := func() (hadBlock bool, err error) {
-		for c.NextBlock() {
-			hadBlock = true
-			if c.Val() == "respawn" {
-				respawn = true
-			} else {
-				return true, c.Err("Expected websocket configuration parameter in block")
-			}
-		}
-		return
-	}
-
-	for c.Next() {
-		var val, path, command string
-
-		// Path or command; not sure which yet
-		if !c.NextArg() {
-			return nil, c.ArgErr()
-		}
-		val = c.Val()
-
-		// Extra configuration may be in a block
-		hadBlock, err := optionalBlock()
-		if err != nil {
-			return nil, err
-		}
-
-		if !hadBlock {
-			// The next argument on this line will be the command or an open curly brace
-			if c.NextArg() {
-				path = val
-				command = c.Val()
-			} else {
-				path = "/"
-				command = val
-			}
-
-			// Okay, check again for optional block
-			hadBlock, err = optionalBlock()
-			if err != nil {
-				return nil, err
-			}
-		}
-
-		// Split command into the actual command and its arguments
-		cmd, args, err := middleware.SplitCommandAndArgs(command)
-		if err != nil {
-			return nil, err
-		}
-
-		websocks = append(websocks, websockets.Config{
-			Path:      path,
-			Command:   cmd,
-			Arguments: args,
-			Respawn:   respawn, // TODO: This isn't used currently
-		})
-	}
-
-	websockets.GatewayInterface = c.AppName + "-CGI/1.1"
-	websockets.ServerSoftware = c.AppName + "/" + c.AppVersion
-
-	return func(next middleware.Handler) middleware.Handler {
-		return websockets.WebSockets{Next: next, Sockets: websocks}
-	}, nil
-}

context.go

@@ -0,0 +1,208 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddy
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"log"
+	"reflect"
+
+	"github.com/mholt/certmagic"
+)
+
+// Context is a type which defines the lifetime of modules that
+// are loaded and provides access to the parent configuration
+// that spawned the modules which are loaded. It should be used
+// with care and wrapped with derivation functions from the
+// standard context package only if you don't need the Caddy
+// specific features. These contexts are cancelled when the
+// lifetime of the modules loaded from it is over.
+//
+// Use NewContext() to get a valid value (but most modules will
+// not actually need to do this).
+type Context struct {
+	context.Context
+	moduleInstances map[string][]interface{}
+	cfg             *Config
+	cleanupFuncs    []func()
+}
+
+// NewContext provides a new context derived from the given
+// context ctx. Normally, you will not need to call this
+// function unless you are loading modules which have a
+// different lifespan than the ones for the context the
+// module was provisioned with. Be sure to call the cancel
+// func when the context is to be cleaned up so that
+// modules which are loaded will be properly unloaded.
+// See standard library context package's documentation.
+func NewContext(ctx Context) (Context, context.CancelFunc) {
+	newCtx := Context{moduleInstances: make(map[string][]interface{}), cfg: ctx.cfg}
+	c, cancel := context.WithCancel(ctx.Context)
+	wrappedCancel := func() {
+		cancel()
+
+		for _, f := range ctx.cleanupFuncs {
+			f()
+		}
+
+		for modName, modInstances := range newCtx.moduleInstances {
+			for _, inst := range modInstances {
+				if cu, ok := inst.(CleanerUpper); ok {
+					err := cu.Cleanup()
+					if err != nil {
+						log.Printf("[ERROR] %s (%p): cleanup: %v", modName, inst, err)
+					}
+				}
+			}
+		}
+	}
+	newCtx.Context = c
+	return newCtx, wrappedCancel
+}
+
+// OnCancel executes f when ctx is cancelled.
+func (ctx *Context) OnCancel(f func()) {
+	ctx.cleanupFuncs = append(ctx.cleanupFuncs, f)
+}
+
+// LoadModule decodes rawMsg into a new instance of mod and
+// returns the value. If mod.New() does not return a pointer
+// value, it is converted to one so that it is unmarshaled
+// into the underlying concrete type. If mod.New is nil, an
+// error is returned. If the module implements Validator or
+// Provisioner interfaces, those methods are invoked to
+// ensure the module is fully configured and valid before
+// being used.
+func (ctx Context) LoadModule(name string, rawMsg json.RawMessage) (interface{}, error) {
+	modulesMu.Lock()
+	mod, ok := modules[name]
+	modulesMu.Unlock()
+	if !ok {
+		return nil, fmt.Errorf("unknown module: %s", name)
+	}
+
+	if mod.New == nil {
+		return nil, fmt.Errorf("module '%s' has no constructor", mod.Name)
+	}
+
+	val := mod.New().(interface{})
+
+	// value must be a pointer for unmarshaling into concrete type, even if
+	// the module's concrete type is a slice or map; New() *should* return
+	// a pointer, otherwise unmarshaling errors or panics will occur
+	if rv := reflect.ValueOf(val); rv.Kind() != reflect.Ptr {
+		log.Printf("[WARNING] ModuleInfo.New() for module '%s' did not return a pointer,"+
+			" so we are using reflection to make a pointer instead; please fix this by"+
+			" using new(Type) or &Type notation in your module's New() function.", name)
+		val = reflect.New(rv.Type()).Elem().Addr().Interface().(Module)
+	}
+
+	// fill in its config only if there is a config to fill in
+	if len(rawMsg) > 0 {
+		err := strictUnmarshalJSON(rawMsg, &val)
+		if err != nil {
+			return nil, fmt.Errorf("decoding module config: %s: %v", mod.Name, err)
+		}
+	}
+
+	if val == nil {
+		// returned module values are almost always type-asserted
+		// before being used, so a nil value would panic; and there
+		// is no good reason to explicitly declare null modules in
+		// a config; it might be because the user is trying to
+		// achieve a result they aren't expecting, which is a smell
+		return nil, fmt.Errorf("module value cannot be null")
+	}
+
+	if prov, ok := val.(Provisioner); ok {
+		err := prov.Provision(ctx)
+		if err != nil {
+			// incomplete provisioning could have left state
+			// dangling, so make sure it gets cleaned up
+			if cleanerUpper, ok := val.(CleanerUpper); ok {
+				err2 := cleanerUpper.Cleanup()
+				if err2 != nil {
+					err = fmt.Errorf("%v; additionally, cleanup: %v", err, err2)
+				}
+			}
+			return nil, fmt.Errorf("provision %s: %v", mod.Name, err)
+		}
+	}
+
+	if validator, ok := val.(Validator); ok {
+		err := validator.Validate()
+		if err != nil {
+			// since the module was already provisioned, make sure we clean up
+			if cleanerUpper, ok := val.(CleanerUpper); ok {
+				err2 := cleanerUpper.Cleanup()
+				if err2 != nil {
+					err = fmt.Errorf("%v; additionally, cleanup: %v", err, err2)
+				}
+			}
+			return nil, fmt.Errorf("%s: invalid configuration: %v", mod.Name, err)
+		}
+	}
+
+	ctx.moduleInstances[name] = append(ctx.moduleInstances[name], val)
+
+	return val, nil
+}
+
+// LoadModuleInline loads a module from a JSON raw message which decodes
+// to a map[string]interface{}, where one of the keys is moduleNameKey
+// and the corresponding value is the module name as a string, which
+// can be found in the given scope.
+//
+// This allows modules to be decoded into their concrete types and
+// used when their names cannot be the unique key in a map, such as
+// when there are multiple instances in the map or it appears in an
+// array (where there are no custom keys). In other words, the key
+// containing the module name is treated special/separate from all
+// the other keys.
+func (ctx Context) LoadModuleInline(moduleNameKey, moduleScope string, raw json.RawMessage) (interface{}, error) {
+	moduleName, raw, err := getModuleNameInline(moduleNameKey, raw)
+	if err != nil {
+		return nil, err
+	}
+
+	val, err := ctx.LoadModule(moduleScope+"."+moduleName, raw)
+	if err != nil {
+		return nil, fmt.Errorf("loading module '%s': %v", moduleName, err)
+	}
+
+	return val, nil
+}
+
+// App returns the configured app named name. If no app with
+// that name is currently configured, a new empty one will be
+// instantiated. (The app module must still be registered.)
+func (ctx Context) App(name string) (interface{}, error) {
+	if app, ok := ctx.cfg.apps[name]; ok {
+		return app, nil
+	}
+	modVal, err := ctx.LoadModule(name, nil)
+	if err != nil {
+		return nil, fmt.Errorf("instantiating new module %s: %v", name, err)
+	}
+	ctx.cfg.apps[name] = modVal.(App)
+	return modVal, nil
+}
+
+// Storage returns the configured Caddy storage implementation.
+func (ctx Context) Storage() certmagic.Storage {
+	return ctx.cfg.storage
+}

dynamicconfig.go

@@ -0,0 +1,358 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddy
+
+import (
+	"bytes"
+	"encoding/json"
+	"fmt"
+	"log"
+	"net/http"
+	"path"
+	"strconv"
+	"strings"
+	"sync"
+)
+
+func init() {
+	RegisterModule(router{})
+}
+
+type router []AdminRoute
+
+// CaddyModule returns the Caddy module information.
+func (router) CaddyModule() ModuleInfo {
+	return ModuleInfo{
+		Name: "admin.routers.dynamic_config",
+		New: func() Module {
+			return router{
+				{
+					Pattern: "/" + rawConfigKey + "/",
+					Handler: http.HandlerFunc(handleConfig),
+				},
+				{
+					Pattern: "/id/",
+					Handler: http.HandlerFunc(handleConfigID),
+				},
+			}
+		},
+	}
+}
+
+func (r router) Routes() []AdminRoute { return r }
+
+// handleConfig handles config changes or exports according to r.
+// This function is safe for concurrent use.
+func handleConfig(w http.ResponseWriter, r *http.Request) {
+	rawCfgMu.Lock()
+	defer rawCfgMu.Unlock()
+	unsyncedHandleConfig(w, r)
+}
+
+// handleConfigID accesses the config through a user-assigned ID
+// that is mapped to its full/expanded path in the JSON structure.
+// It is the same as handleConfig except it replaces the ID in
+// the request path with the full, expanded URL path.
+// This function is safe for concurrent use.
+func handleConfigID(w http.ResponseWriter, r *http.Request) {
+	parts := strings.Split(r.URL.Path, "/")
+	if len(parts) < 3 || parts[2] == "" {
+		http.Error(w, "request path is missing object ID", http.StatusBadRequest)
+		return
+	}
+	id := parts[2]
+
+	rawCfgMu.Lock()
+	defer rawCfgMu.Unlock()
+
+	// map the ID to the expanded path
+	expanded, ok := rawCfgIndex[id]
+	if !ok {
+		http.Error(w, "unknown object ID: "+id, http.StatusBadRequest)
+		return
+	}
+
+	// piece the full URL path back together
+	parts = append([]string{expanded}, parts[3:]...)
+	r.URL.Path = path.Join(parts...)
+
+	unsyncedHandleConfig(w, r)
+}
+
+// configIndex recurisvely searches ptr for object fields named "@id"
+// and maps that ID value to the full configPath in the index.
+// This function is NOT safe for concurrent access; use rawCfgMu.
+func configIndex(ptr interface{}, configPath string, index map[string]string) error {
+	switch val := ptr.(type) {
+	case map[string]interface{}:
+		for k, v := range val {
+			if k == "@id" {
+				switch idVal := v.(type) {
+				case string:
+					index[idVal] = configPath
+				case float64: // all JSON numbers decode as float64
+					index[fmt.Sprintf("%v", idVal)] = configPath
+				default:
+					return fmt.Errorf("%s: @id field must be a string or number", configPath)
+				}
+				delete(val, "@id") // field is no longer needed, and will break config if not removed
+				continue
+			}
+			// traverse this object property recursively
+			err := configIndex(val[k], path.Join(configPath, k), index)
+			if err != nil {
+				return err
+			}
+		}
+	case []interface{}:
+		// traverse each element of the array recursively
+		for i := range val {
+			err := configIndex(val[i], path.Join(configPath, strconv.Itoa(i)), index)
+			if err != nil {
+				return err
+			}
+		}
+	}
+
+	return nil
+}
+
+// unsycnedHandleConfig handles config accesses without a lock
+// on rawCfgMu. This is NOT safe for concurrent use, so be sure
+// to acquire a lock on rawCfgMu before calling this.
+func unsyncedHandleConfig(w http.ResponseWriter, r *http.Request) {
+	switch r.Method {
+	case http.MethodGet, http.MethodPost, http.MethodPut, http.MethodPatch, http.MethodDelete:
+	default:
+		http.Error(w, http.StatusText(http.StatusMethodNotAllowed), http.StatusMethodNotAllowed)
+		return
+	}
+
+	// perform the mutation with our decoded representation
+	// (the map), which may change pointers deep within it
+	err := mutateConfig(w, r)
+	if err != nil {
+		http.Error(w, "mutating config: "+err.Error(), http.StatusBadRequest)
+		return
+	}
+
+	if r.Method != http.MethodGet {
+		// find any IDs in this config and index them
+		idx := make(map[string]string)
+		err = configIndex(rawCfg[rawConfigKey], "/config", idx)
+		if err != nil {
+			http.Error(w, "indexing config: "+err.Error(), http.StatusInternalServerError)
+			return
+		}
+
+		// the mutation is complete, so encode the entire config as JSON
+		newCfg, err := json.Marshal(rawCfg[rawConfigKey])
+		if err != nil {
+			http.Error(w, "encoding new config: "+err.Error(), http.StatusBadRequest)
+			return
+		}
+
+		// if nothing changed, no need to do a whole reload unless the client forces it
+		if r.Header.Get("Cache-Control") != "must-revalidate" && bytes.Equal(rawCfgJSON, newCfg) {
+			log.Printf("[ADMIN][INFO] Config is unchanged")
+			return
+		}
+
+		// load this new config; if it fails, we need to revert to
+		// our old representation of caddy's actual config
+		err = Load(bytes.NewReader(newCfg))
+		if err != nil {
+			// restore old config state to keep it consistent
+			// with what caddy is still running; we need to
+			// unmarshal it again because it's likely that
+			// pointers deep in our rawCfg map were modified
+			var oldCfg interface{}
+			err2 := json.Unmarshal(rawCfgJSON, &oldCfg)
+			if err2 != nil {
+				err = fmt.Errorf("%v; additionally, restoring old config: %v", err, err2)
+			}
+			rawCfg[rawConfigKey] = oldCfg
+
+			// report error
+			log.Printf("[ADMIN][ERROR] loading config: %v", err)
+			http.Error(w, err.Error(), http.StatusBadRequest)
+			return
+		}
+
+		// success, so update our stored copy of the encoded
+		// config to keep it consistent with what caddy is now
+		// running (storing an encoded copy is not strictly
+		// necessary, but avoids an extra json.Marshal for
+		// each config change)
+		rawCfgJSON = newCfg
+		rawCfgIndex = idx
+	}
+}
+
+// mutateConfig changes the rawCfg according to r. It is NOT
+// safe for concurrent use; use rawCfgMu. If the request's
+// method is GET, the config will not be changed.
+func mutateConfig(w http.ResponseWriter, r *http.Request) error {
+	var err error
+	var val interface{}
+
+	// if there is a request body, make sure we recognize its content-type and decode it
+	if r.Method != http.MethodGet && r.Method != http.MethodDelete {
+		if ct := r.Header.Get("Content-Type"); !strings.Contains(ct, "/json") {
+			return fmt.Errorf("unacceptable content-type: %v; 'application/json' required", ct)
+		}
+		err = json.NewDecoder(r.Body).Decode(&val)
+		if err != nil {
+			return fmt.Errorf("decoding request body: %v", err)
+		}
+	}
+
+	buf := new(bytes.Buffer)
+	enc := json.NewEncoder(buf)
+
+	cleanPath := strings.Trim(r.URL.Path, "/")
+	if cleanPath == "" {
+		return fmt.Errorf("no traversable path")
+	}
+
+	parts := strings.Split(cleanPath, "/")
+	if len(parts) == 0 {
+		return fmt.Errorf("path missing")
+	}
+
+	var ptr interface{} = rawCfg
+
+traverseLoop:
+	for i, part := range parts {
+		switch v := ptr.(type) {
+		case map[string]interface{}:
+			// if the next part enters a slice, and the slice is our destination,
+			// handle it specially (because appending to the slice copies the slice
+			// header, which does not replace the original one like we want)
+			if arr, ok := v[part].([]interface{}); ok && i == len(parts)-2 {
+				var idx int
+				if r.Method != http.MethodPost {
+					idxStr := parts[len(parts)-1]
+					idx, err = strconv.Atoi(idxStr)
+					if err != nil {
+						return fmt.Errorf("[%s] invalid array index '%s': %v",
+							r.URL.Path, idxStr, err)
+					}
+					if idx < 0 || idx >= len(arr) {
+						return fmt.Errorf("[%s] array index out of bounds: %s", r.URL.Path, idxStr)
+					}
+				}
+
+				switch r.Method {
+				case http.MethodGet:
+					err = enc.Encode(arr[idx])
+					if err != nil {
+						return fmt.Errorf("encoding config: %v", err)
+					}
+				case http.MethodPost:
+					v[part] = append(arr, val)
+				case http.MethodPut:
+					// avoid creation of new slice and a second copy (see
+					// https://github.com/golang/go/wiki/SliceTricks#insert)
+					arr = append(arr, nil)
+					copy(arr[idx+1:], arr[idx:])
+					arr[idx] = val
+					v[part] = arr
+				case http.MethodPatch:
+					arr[idx] = val
+				case http.MethodDelete:
+					v[part] = append(arr[:idx], arr[idx+1:]...)
+				default:
+					return fmt.Errorf("unrecognized method %s", r.Method)
+				}
+				break traverseLoop
+			}
+
+			if i == len(parts)-1 {
+				switch r.Method {
+				case http.MethodGet:
+					err = enc.Encode(v[part])
+					if err != nil {
+						return fmt.Errorf("encoding config: %v", err)
+					}
+				case http.MethodPost:
+					if arr, ok := v[part].([]interface{}); ok {
+						// if the part is an existing list, POST appends to it
+						// TODO: Do we ever reach this point, since we handle arrays
+						// separately above?
+						v[part] = append(arr, val)
+					} else {
+						// otherwise, it simply sets the value
+						v[part] = val
+					}
+				case http.MethodPut:
+					if _, ok := v[part]; ok {
+						return fmt.Errorf("[%s] key already exists: %s", r.URL.Path, part)
+					}
+					v[part] = val
+				case http.MethodPatch:
+					if _, ok := v[part]; !ok {
+						return fmt.Errorf("[%s] key does not exist: %s", r.URL.Path, part)
+					}
+					v[part] = val
+				case http.MethodDelete:
+					delete(v, part)
+				default:
+					return fmt.Errorf("unrecognized method %s", r.Method)
+				}
+			} else {
+				ptr = v[part]
+			}
+
+		case []interface{}:
+			partInt, err := strconv.Atoi(part)
+			if err != nil {
+				return fmt.Errorf("[/%s] invalid array index '%s': %v",
+					strings.Join(parts[:i+1], "/"), part, err)
+			}
+			if partInt < 0 || partInt >= len(v) {
+				return fmt.Errorf("[/%s] array index out of bounds: %s",
+					strings.Join(parts[:i+1], "/"), part)
+			}
+			ptr = v[partInt]
+
+		default:
+			return fmt.Errorf("invalid path: %s", parts[:i+1])
+		}
+	}
+
+	if r.Method == http.MethodGet {
+		w.Header().Set("Content-Type", "application/json")
+		w.Write(buf.Bytes())
+	}
+
+	return nil
+}
+
+var (
+	// rawCfg is the current, generic-decoded configuration;
+	// we initialize it as a map with one field ("config")
+	// to maintain parity with the API endpoint and to avoid
+	// the special case of having to access/mutate the variable
+	// directly without traversing into it
+	rawCfg = map[string]interface{}{
+		rawConfigKey: nil,
+	}
+	rawCfgJSON  []byte            // keeping the encoded form avoids an extra Marshal on changes
+	rawCfgIndex map[string]string // map of user-assigned ID to expanded path
+	rawCfgMu    sync.Mutex        // protects rawCfg, rawCfgJSON, and rawCfgIndex
+)
+
+const rawConfigKey = "config"

dynamicconfig_test.go

@@ -0,0 +1,123 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddy
+
+import (
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"reflect"
+	"strings"
+	"testing"
+)
+
+func TestMutateConfig(t *testing.T) {
+	// each test is performed in sequence, so
+	// each change builds on the previous ones;
+	// the config is not reset between tests
+	for i, tc := range []struct {
+		method    string
+		path      string // rawConfigKey will be prepended
+		payload   string
+		expect    string // JSON representation of what the whole config is expected to be after the request
+		shouldErr bool
+	}{
+		{
+			method:  "POST",
+			path:    "",
+			payload: `{"foo": "bar", "list": ["a", "b", "c"]}`, // starting value
+			expect:  `{"foo": "bar", "list": ["a", "b", "c"]}`,
+		},
+		{
+			method:  "POST",
+			path:    "/foo",
+			payload: `"jet"`,
+			expect:  `{"foo": "jet", "list": ["a", "b", "c"]}`,
+		},
+		{
+			method:  "POST",
+			path:    "/bar",
+			payload: `{"aa": "bb", "qq": "zz"}`,
+			expect:  `{"foo": "jet", "bar": {"aa": "bb", "qq": "zz"}, "list": ["a", "b", "c"]}`,
+		},
+		{
+			method: "DELETE",
+			path:   "/bar/qq",
+			expect: `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c"]}`,
+		},
+		{
+			method:  "POST",
+			path:    "/list",
+			payload: `"e"`,
+			expect:  `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c", "e"]}`,
+		},
+		{
+			method:  "PUT",
+			path:    "/list/3",
+			payload: `"d"`,
+			expect:  `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c", "d", "e"]}`,
+		},
+		{
+			method: "DELETE",
+			path:   "/list/3",
+			expect: `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c", "e"]}`,
+		},
+		{
+			method:  "PATCH",
+			path:    "/list/3",
+			payload: `"d"`,
+			expect:  `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c", "d"]}`,
+		},
+	} {
+		req, err := http.NewRequest(tc.method, rawConfigKey+tc.path, strings.NewReader(tc.payload))
+		if err != nil {
+			t.Fatalf("Test %d: making test request: %v", i, err)
+		}
+		req.Header.Set("Content-Type", "application/json")
+
+		w := httptest.NewRecorder()
+
+		err = mutateConfig(w, req)
+
+		if tc.shouldErr && err == nil {
+			t.Fatalf("Test %d: Expected error return value, but got: %v", i, err)
+		}
+		if !tc.shouldErr && err != nil {
+			t.Fatalf("Test %d: Should not have had error return value, but got: %v", i, err)
+		}
+
+		if tc.shouldErr && w.Code == http.StatusOK {
+			t.Fatalf("Test %d: Expected error, but got HTTP %d: %s",
+				i, w.Code, w.Body.String())
+		}
+		if !tc.shouldErr && w.Code != http.StatusOK {
+			t.Fatalf("Test %d: Should not have errored, but got HTTP %d: %s",
+				i, w.Code, w.Body.String())
+		}
+
+		// decode the expected config so we can do a convenient DeepEqual
+		var expectedDecoded interface{}
+		err = json.Unmarshal([]byte(tc.expect), &expectedDecoded)
+		if err != nil {
+			t.Fatalf("Test %d: Unmarshaling expected config: %v", i, err)
+		}
+
+		// make sure the resulting config is as we expect it
+		if !reflect.DeepEqual(rawCfg[rawConfigKey], expectedDecoded) {
+			t.Fatalf("Test %d:\nExpected:\n\t%#v\nActual:\n\t%#v",
+				i, expectedDecoded, rawCfg[rawConfigKey])
+		}
+	}
+}

go.mod

@@ -0,0 +1,27 @@
+module github.com/caddyserver/caddy/v2
+
+go 1.13
+
+require (
+	github.com/Masterminds/sprig/v3 v3.0.0
+	github.com/andybalholm/brotli v0.0.0-20190821151343-b60f0d972eeb
+	github.com/dustin/go-humanize v1.0.0
+	github.com/go-acme/lego/v3 v3.1.0
+	github.com/golang/groupcache v0.0.0-20191002201903-404acd9df4cc
+	github.com/ilibs/json5 v1.0.1
+	github.com/imdario/mergo v0.3.8 // indirect
+	github.com/keybase/go-ps v0.0.0-20190827175125-91aafc93ba19
+	github.com/klauspost/compress v1.8.6
+	github.com/klauspost/cpuid v1.2.1
+	github.com/lucas-clemente/quic-go v0.12.1
+	github.com/mholt/certmagic v0.7.4
+	github.com/muhammadmuzzammil1998/jsonc v0.0.0-20190906142622-1265e9b150c6
+	github.com/rs/cors v1.7.0
+	github.com/russross/blackfriday/v2 v2.0.1
+	github.com/starlight-go/starlight v0.0.0-20181207205707-b06f321544f3
+	github.com/vulcand/oxy v1.0.0
+	go.starlark.net v0.0.0-20190919145610-979af19b165c
+	golang.org/x/crypto v0.0.0-20191010185427-af544f31c8ac
+	golang.org/x/net v0.0.0-20191009170851-d66e71096ffb
+	golang.org/x/time v0.0.0-20190921001708-c4c64cad1fd0
+)

go.sum

@@ -0,0 +1,387 @@
+cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
+cloud.google.com/go v0.38.0/go.mod h1:990N+gfupTy94rShfmMCWGDn0LpTmnzTp2qbd1dvSRU=
+contrib.go.opencensus.io/exporter/ocagent v0.4.12/go.mod h1:450APlNTSR6FrvC3CTRqYosuDstRB9un7SOx2k/9ckA=
+github.com/Azure/azure-sdk-for-go v32.4.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
+github.com/Azure/go-autorest/autorest v0.1.0/go.mod h1:AKyIcETwSUFxIcs/Wnq/C+kwCtlEYGUVd7FPNb2slmg=
+github.com/Azure/go-autorest/autorest v0.5.0/go.mod h1:9HLKlQjVBH6U3oDfsXOeVc56THsLPw1L03yban4xThw=
+github.com/Azure/go-autorest/autorest/adal v0.1.0/go.mod h1:MeS4XhScH55IST095THyTxElntu7WqB7pNbZo8Q5G3E=
+github.com/Azure/go-autorest/autorest/adal v0.2.0/go.mod h1:MeS4XhScH55IST095THyTxElntu7WqB7pNbZo8Q5G3E=
+github.com/Azure/go-autorest/autorest/azure/auth v0.1.0/go.mod h1:Gf7/i2FUpyb/sGBLIFxTBzrNzBo7aPXXE3ZVeDRwdpM=
+github.com/Azure/go-autorest/autorest/azure/cli v0.1.0/go.mod h1:Dk8CUAt/b/PzkfeRsWzVG9Yj3ps8mS8ECztu43rdU8U=
+github.com/Azure/go-autorest/autorest/date v0.1.0/go.mod h1:plvfp3oPSKwf2DNjlBjWF/7vwR+cUD/ELuzDCXwHUVA=
+github.com/Azure/go-autorest/autorest/mocks v0.1.0/go.mod h1:OTyCOPRA2IgIlWxVYxBee2F5Gr4kF2zd2J5cFRaIDN0=
+github.com/Azure/go-autorest/autorest/to v0.2.0/go.mod h1:GunWKJp1AEqgMaGLV+iocmRAJWqST1wQYhyyjXJ3SJc=
+github.com/Azure/go-autorest/autorest/validation v0.1.0/go.mod h1:Ha3z/SqBeaalWQvokg3NZAlQTalVMtOIAs1aGK7G6u8=
+github.com/Azure/go-autorest/logger v0.1.0/go.mod h1:oExouG+K6PryycPJfVSxi/koC6LSNgds39diKLz7Vrc=
+github.com/Azure/go-autorest/tracing v0.1.0/go.mod h1:ROEEAFwXycQw7Sn3DXNtEedEvdeRAgDr0izn4z5Ij88=
+github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
+github.com/Masterminds/goutils v1.1.0 h1:zukEsf/1JZwCMgHiK3GZftabmxiCw4apj3a28RPBiVg=
+github.com/Masterminds/goutils v1.1.0/go.mod h1:8cTjp+g8YejhMuvIA5y2vz3BpJxksy863GQaJW2MFNU=
+github.com/Masterminds/semver/v3 v3.0.1 h1:2kKm5lb7dKVrt5TYUiAavE6oFc1cFT0057UVGT+JqLk=
+github.com/Masterminds/semver/v3 v3.0.1/go.mod h1:VPu/7SZ7ePZ3QOrcuXROw5FAcLl4a0cBrbBpGY/8hQs=
+github.com/Masterminds/sprig/v3 v3.0.0 h1:KSQz7Nb08/3VU9E4ns29dDxcczhOD1q7O1UfM4G3t3g=
+github.com/Masterminds/sprig/v3 v3.0.0/go.mod h1:NEUY/Qq8Gdm2xgYA+NwJM6wmfdRV9xkh8h/Rld20R0U=
+github.com/OpenDNS/vegadns2client v0.0.0-20180418235048-a3fa4a771d87/go.mod h1:iGLljf5n9GjT6kc0HBvyI1nOKnGQbNB66VzSNbK5iks=
+github.com/Shopify/sarama v1.19.0/go.mod h1:FVkBWblsNy7DGZRfXLU0O9RCGt5g3g3yEuWXgklEdEo=
+github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI=
+github.com/akamai/AkamaiOPEN-edgegrid-golang v0.9.0/go.mod h1:zpDJeKyp9ScW4NNrbdr+Eyxvry3ilGPewKoXw3XGN1k=
+github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
+github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
+github.com/aliyun/alibaba-cloud-sdk-go v0.0.0-20190808125512-07798873deee/go.mod h1:myCDvQSzCW+wB1WAlocEru4wMGJxy+vlxHdhegi1CDQ=
+github.com/aliyun/aliyun-oss-go-sdk v0.0.0-20190307165228-86c17b95fcd5/go.mod h1:T/Aws4fEfogEE9v+HPhhw+CntffsBHJ8nXQCwKr0/g8=
+github.com/andybalholm/brotli v0.0.0-20190821151343-b60f0d972eeb h1:ZSlUsEd11C/uRzhZHOgANARJ03fkwmjJEa6g2Cqjlo4=
+github.com/andybalholm/brotli v0.0.0-20190821151343-b60f0d972eeb/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
+github.com/apache/thrift v0.12.0/go.mod h1:cp2SuWMxlEZw2r+iP2GNCdIi4C1qmUzdZFSVb+bacwQ=
+github.com/aws/aws-sdk-go v1.23.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
+github.com/baiyubin/aliyun-sts-go-sdk v0.0.0-20180326062324-cfa1a18b161f/go.mod h1:AuiFmCCPBSrqvVMvuqFuk0qogytodnVFVSN5CeJB8Gc=
+github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
+github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
+github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
+github.com/cenkalti/backoff/v3 v3.0.0 h1:ske+9nBpD9qZsTBoF41nW5L+AIuFBKMeze18XQ3eG1c=
+github.com/cenkalti/backoff/v3 v3.0.0/go.mod h1:cIeZDE3IrqwwJl6VUwCN6trj1oXrTS4rc0ij+ULvLYs=
+github.com/census-instrumentation/opencensus-proto v0.2.0/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU=
+github.com/cheekybits/genny v1.0.0 h1:uGGa4nei+j20rOSeDeP5Of12XVm7TGUd4dJA9RDitfE=
+github.com/cheekybits/genny v1.0.0/go.mod h1:+tQajlRqAUrPI7DOSpB0XAqZYtQakVtB7wXkRAgjxjQ=
+github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
+github.com/cloudflare/cloudflare-go v0.10.0/go.mod h1:fOESqHl/jzAmCtEyjceLkw3v0rVjzl8V9iehxZPynXY=
+github.com/cloudflare/cloudflare-go v0.10.2/go.mod h1:qhVI5MKwBGhdNU89ZRz2plgYutcJ5PCekLxXn56w6SY=
+github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd h1:qMd81Ts1T2OTKmB4acZcyKaMtRnY5Y44NuXGX2GFJ1w=
+github.com/codahale/hdrhistogram v0.0.0-20161010025455-3a0bb77429bd/go.mod h1:sE/e/2PUdi/liOCUjSTXgM1o87ZssimdTWN964YiIeI=
+github.com/cpu/goacmedns v0.0.1/go.mod h1:sesf/pNnCYwUevQEQfEwY0Y3DydlQWSGZbaMElOWxok=
+github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/decker502/dnspod-go v0.2.0/go.mod h1:qsurYu1FgxcDwfSwXJdLt4kRsBLZeosEb9uq4Sy+08g=
+github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
+github.com/dimchansky/utfbom v1.1.0/go.mod h1:rO41eb7gLfo8SF1jd9F8HplJm1Fewwi4mQvIirEdv+8=
+github.com/dnaeon/go-vcr v0.0.0-20180814043457-aafff18a5cc2/go.mod h1:aBB1+wY4s93YsC3HHjMBMrwTj2R9FHDzUr9KyGc8n1E=
+github.com/dnsimple/dnsimple-go v0.30.0/go.mod h1:O5TJ0/U6r7AfT8niYNlmohpLbCSG+c71tQlGr9SeGrg=
+github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo=
+github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
+github.com/eapache/go-resiliency v1.1.0/go.mod h1:kFI+JgMyC7bLPUVY133qvEBtVayf5mFgVsvEsIPBvNs=
+github.com/eapache/go-xerial-snappy v0.0.0-20180814174437-776d5712da21/go.mod h1:+020luEh2TKB4/GOp8oxxtq0Daoen/Cii55CzbTV6DU=
+github.com/eapache/queue v1.1.0/go.mod h1:6eCeP0CKFpHLu8blIFXhExK/dRa7WDZfr6jVFPTqq+I=
+github.com/exoscale/egoscale v0.18.1/go.mod h1:Z7OOdzzTOz1Q1PjQXumlz9Wn/CddH0zSYdCF3rnBKXE=
+github.com/fatih/structs v1.1.0/go.mod h1:9NiDSp5zOcgEDl+j00MP/WkGVPOlPRLejGD8Ga6PJ7M=
+github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV9I=
+github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
+github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
+github.com/go-acme/lego/v3 v3.0.2/go.mod h1:sMoLjf8BUo4Jexg+6Xw5KeFx98KVZ7Nfczh9tzLyhJU=
+github.com/go-acme/lego/v3 v3.1.0 h1:yanYFoYW8azFkCvJfIk7edWWfjkYkhDxe45ZsxoW4Xk=
+github.com/go-acme/lego/v3 v3.1.0/go.mod h1:074uqt+JS6plx+c9Xaiz6+L+GBb+7itGtzfcDM2AhEE=
+github.com/go-cmd/cmd v1.0.5/go.mod h1:y8q8qlK5wQibcw63djSl/ntiHUHXHGdCkPk0j4QeW4s=
+github.com/go-errors/errors v1.0.1/go.mod h1:f4zRHt4oKfwPJE5k8C9vpYG+aDHdBFUsgrm6/TyX73Q=
+github.com/go-ini/ini v1.44.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
+github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
+github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
+github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
+github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
+github.com/gofrs/uuid v3.2.0+incompatible/go.mod h1:b2aQJv3Z4Fp6yNu3cdSllBxTCLRxnplIgP/c0N/04lM=
+github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
+github.com/gogo/protobuf v1.2.0/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
+github.com/goji/httpauth v0.0.0-20160601135302-2da839ab0f4d/go.mod h1:nnjvkQ9ptGaCkuDUx6wNykzzlUixGxvkme+H/lnzb+A=
+github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
+github.com/golang/groupcache v0.0.0-20191002201903-404acd9df4cc h1:55rEp52jU6bkyslZ1+C/7NGfpQsEc6pxGLAGDOctqbw=
+github.com/golang/groupcache v0.0.0-20191002201903-404acd9df4cc/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
+github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
+github.com/golang/mock v1.2.0 h1:28o5sBqPkBsMGnC6b4MvE2TzSr5/AT4c/1fLqVGIwlk=
+github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
+github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/protobuf v1.3.0/go.mod h1:Qd/q+1AKNOZr9uGQzbzCmRO6sUih6GTPZv6a1/R87v0=
+github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/protobuf v1.3.2 h1:6nsPYzhq5kReh6QImI3k5qWzO4PEbvbIW2cwSfR/6xs=
+github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
+github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
+github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
+github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
+github.com/google/go-cmp v0.3.0 h1:crn/baboCvb5fXaQ0IJ1SGTsTVrWpDsCWC8EGETZijY=
+github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
+github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
+github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
+github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs=
+github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57/go.mod h1:zfwlbNMJ+OItoe0UupaVj+oy1omPYYDuagoSzA8v9mc=
+github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=
+github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
+github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
+github.com/gophercloud/gophercloud v0.3.0/go.mod h1:vxM41WHh5uqHVBMZHzuwNOHh8XEoIEcSTewFxm1c5g8=
+github.com/gopherjs/gopherjs v0.0.0-20181017120253-0766667cb4d1/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
+github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
+github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
+github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
+github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
+github.com/gravitational/trace v0.0.0-20190726142706-a535a178675f/go.mod h1:RvdOUHE4SHqR3oXlFFKnGzms8a5dugHygGw1bqDstYI=
+github.com/grpc-ecosystem/grpc-gateway v1.8.5/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
+github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542/go.mod h1:Ow0tF8D4Kplbc8s8sSb3V2oUCygFHVp8gC3Dn6U4MNI=
+github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
+github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
+github.com/hashicorp/golang-lru v0.5.3/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4=
+github.com/hpcloud/tail v1.0.0 h1:nfCOvKYfkgYP8hkirhJocXT2+zOD8yUNjXaWfTlyFKI=
+github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
+github.com/huandu/xstrings v1.2.0 h1:yPeWdRnmynF7p+lLYz0H2tthW9lqhMJrQV/U7yy4wX0=
+github.com/huandu/xstrings v1.2.0/go.mod h1:DvyZB1rfVYsBIigL8HwpZgxHwXozlTgGqn63UyNX5k4=
+github.com/iij/doapi v0.0.0-20190504054126-0bbf12d6d7df/go.mod h1:QMZY7/J/KSQEhKWFeDesPjMj+wCHReeknARU3wqlyN4=
+github.com/ilibs/json5 v1.0.1 h1:3e14wUQM8PyK6Hf1bM+zAQFxfG+N5oZj35x5vCNeQ58=
+github.com/ilibs/json5 v1.0.1/go.mod h1:kXsGuzHMPuZZTN15l0IQzy5PR8DrDhPB24tFgwpdKME=
+github.com/imdario/mergo v0.3.7/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
+github.com/imdario/mergo v0.3.8 h1:CGgOkSJeqMRmt0D9XLWExdT4m4F1vd3FV3VPt+0VxkQ=
+github.com/imdario/mergo v0.3.8/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
+github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
+github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
+github.com/json-iterator/go v1.1.5/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
+github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
+github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
+github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
+github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
+github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
+github.com/keybase/go-ps v0.0.0-20190827175125-91aafc93ba19 h1:WjT3fLi9n8YWh/Ih8Q1LHAPsTqGddPcHqscN+PJ3i68=
+github.com/keybase/go-ps v0.0.0-20190827175125-91aafc93ba19/go.mod h1:hY+WOq6m2FpbvyrI93sMaypsttvaIL5nhVR92dTMUcQ=
+github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
+github.com/klauspost/compress v1.8.6 h1:970MQcQdxX7hfgc/aqmB4a3grW0ivUVV6i1TLkP8CiE=
+github.com/klauspost/compress v1.8.6/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
+github.com/klauspost/cpuid v1.2.0/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
+github.com/klauspost/cpuid v1.2.1 h1:vJi+O/nMdFt0vqm8NZBI6wzALWdA2X+egi0ogNyrC/w=
+github.com/klauspost/cpuid v1.2.1/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
+github.com/kolo/xmlrpc v0.0.0-20190717152603-07c4ee3fd181/go.mod h1:o03bZfuBwAXHetKXuInt4S7omeXUu62/A845kiycsSQ=
+github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
+github.com/konsorten/go-windows-terminal-sequences v1.0.2 h1:DB17ag19krx9CFsz4o3enTrPXyIXCl+2iCXH/aMAp9s=
+github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
+github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
+github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
+github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
+github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
+github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
+github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
+github.com/labbsr0x/bindman-dns-webhook v1.0.0/go.mod h1:pn4jcNjxSywRWDPDyGkFzgSnwty18OFdiUFc6S6fpgc=
+github.com/labbsr0x/bindman-dns-webhook v1.0.2/go.mod h1:p6b+VCXIR8NYKpDr8/dg1HKfQoRHCdcsROXKvmoehKA=
+github.com/labbsr0x/goh v0.0.0-20190417202808-8b16b4848295/go.mod h1:RBxeaayaaMmp7GxwHiKANjkg9e+rxjOm4mB5vD5rt/I=
+github.com/labbsr0x/goh v1.0.1/go.mod h1:8K2UhVoaWXcCU7Lxoa2omWnC8gyW8px7/lmO61c027w=
+github.com/linode/linodego v0.10.0/go.mod h1:cziNP7pbvE3mXIPneHj0oRY8L1WtGEIKlZ8LANE4eXA=
+github.com/liquidweb/liquidweb-go v1.6.0/go.mod h1:UDcVnAMDkZxpw4Y7NOHkqoeiGacVLEIG/i5J9cyixzQ=
+github.com/lucas-clemente/quic-go v0.12.1 h1:BPITli+6KnKogtTxBk2aS4okr5dUHz2LtIDAP1b8UL4=
+github.com/lucas-clemente/quic-go v0.12.1/go.mod h1:UXJJPE4RfFef/xPO5wQm0tITK8gNfqwTxjbE7s3Vb8s=
+github.com/mailgun/minheap v0.0.0-20170619185613-3dbe6c6bf55f/go.mod h1:V3EvCedtJTvUYzJF2GZMRB0JMlai+6cBu3VCTQz33GQ=
+github.com/mailgun/multibuf v0.0.0-20150714184110-565402cd71fb/go.mod h1:E0vRBBIQUHcRtmL/oR6w/jehh4FJqJFxe86gBnw9gXc=
+github.com/mailgun/timetools v0.0.0-20141028012446-7e6055773c51 h1:Kg/NPZLLC3aAFr1YToMs98dbCdhootQ1hZIvZU28hAQ=
+github.com/mailgun/timetools v0.0.0-20141028012446-7e6055773c51/go.mod h1:RYmqHbhWwIz3z9eVmQ2rx82rulEMG0t+Q1bzfc9DYN4=
+github.com/mailgun/ttlmap v0.0.0-20170619185759-c1c17f74874f/go.mod h1:8heskWJ5c0v5J9WH89ADhyal1DOZcayll8fSbhB+/9A=
+github.com/marten-seemann/qpack v0.1.0 h1:/0M7lkda/6mus9B8u34Asqm8ZhHAAt9Ho0vniNuVSVg=
+github.com/marten-seemann/qpack v0.1.0/go.mod h1:LFt1NU/Ptjip0C2CPkhimBz5CGE3WGDAUWqna+CNTrI=
+github.com/marten-seemann/qtls v0.3.2 h1:O7awy4bHEzSX/K3h+fZig3/Vo03s/RxlxgsAk9sYamI=
+github.com/marten-seemann/qtls v0.3.2/go.mod h1:xzjG7avBwGGbdZ8dTGxlBnLArsVKLvwmjgmPuiQEcYk=
+github.com/mattn/go-isatty v0.0.3/go.mod h1:M+lRXTBqGeGNdLjl/ufCoiOlB5xdOkqRJdNxMWT7Zi4=
+github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
+github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
+github.com/mattn/go-tty v0.0.0-20180219170247-931426f7535a/go.mod h1:XPvLUNfbS4fJH25nqRHfWLMa1ONC8Amw+mIA639KxkE=
+github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
+github.com/miekg/dns v1.1.15 h1:CSSIDtllwGLMoA6zjdKnaE6Tx6eVUxQ29LUgGetiDCI=
+github.com/miekg/dns v1.1.15/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
+github.com/mitchellh/copystructure v1.0.0 h1:Laisrj+bAB6b/yJwB5Bt3ITZhGJdqmxquMKeZ+mmkFQ=
+github.com/mitchellh/copystructure v1.0.0/go.mod h1:SNtv71yrdKgLRyLFxmLdkAbkKEFWgYaq1OVrnRcwhnw=
+github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
+github.com/mitchellh/go-vnc v0.0.0-20150629162542-723ed9867aed/go.mod h1:3rdaFaCv4AyBgu5ALFM0+tSuHrBh6v692nyQe3ikrq0=
+github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
+github.com/mitchellh/reflectwalk v1.0.0 h1:9D+8oIskB4VJBN5SFlmc27fSlIBZaov1Wpk/IfikLNY=
+github.com/mitchellh/reflectwalk v1.0.0/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
+github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
+github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
+github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
+github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
+github.com/muhammadmuzzammil1998/jsonc v0.0.0-20190906142622-1265e9b150c6 h1:EajWCEv0scxMWyMHWxJbFK70brsPIl4TLQJ0zaOeOiI=
+github.com/muhammadmuzzammil1998/jsonc v0.0.0-20190906142622-1265e9b150c6/go.mod h1:saF2fIVw4banK0H4+/EuqfFLpRnoy5S+ECwTOCcRcSU=
+github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
+github.com/namedotcom/go v0.0.0-20180403034216-08470befbe04/go.mod h1:5sN+Lt1CaY4wsPvgQH/jsuJi4XO2ssZbdsIizr4CVC8=
+github.com/nbio/st v0.0.0-20140626010706-e9e8d9816f32/go.mod h1:9wM+0iRr9ahx58uYLpLIr5fm8diHn0JbqRycJi6w0Ms=
+github.com/nrdcg/auroradns v1.0.0/go.mod h1:6JPXKzIRzZzMqtTDgueIhTi6rFf1QvYE/HzqidhOhjw=
+github.com/nrdcg/goinwx v0.6.1/go.mod h1:XPiut7enlbEdntAqalBIqcYcTEVhpv/dKWgDCX2SwKQ=
+github.com/nrdcg/namesilo v0.2.1/go.mod h1:lwMvfQTyYq+BbjJd30ylEG4GPSS6PII0Tia4rRpRiyw=
+github.com/olekukonko/tablewriter v0.0.1/go.mod h1:vsDQFd/mU46D+Z4whnwzcISnGGzXWMclvtLoiIKAKIo=
+github.com/onsi/ginkgo v1.6.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
+github.com/onsi/ginkgo v1.7.0 h1:WSHQ+IS43OoUrWtD1/bbclrwK8TTH5hzp+umCiuxHgs=
+github.com/onsi/ginkgo v1.7.0/go.mod h1:lLunBs/Ym6LB5Z9jYTR76FiuTmxDTDusOGeTQH+WWjE=
+github.com/onsi/gomega v1.4.3 h1:RE1xgDvH7imwFD45h+u2SgIfERHlS2yNG4DObb5BSKU=
+github.com/onsi/gomega v1.4.3/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
+github.com/openzipkin/zipkin-go v0.1.6/go.mod h1:QgAqvLzwWbR/WpD4A3cGpPtJrZXNIiJc5AZX7/PBEpw=
+github.com/oracle/oci-go-sdk v7.0.0+incompatible/go.mod h1:VQb79nF8Z2cwLkLS35ukwStZIg5F66tcBccjip/j888=
+github.com/ovh/go-ovh v0.0.0-20181109152953-ba5adb4cf014/go.mod h1:joRatxRJaZBsY3JAOEMcoOp05CnZzsx4scTxi95DHyQ=
+github.com/pierrec/lz4 v2.0.5+incompatible/go.mod h1:pdkljMzZIN41W+lC3N2tnIh5sFi+IEE17M5jbnwPHcY=
+github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
+github.com/prometheus/client_golang v0.9.3-0.20190127221311-3c4408c8b829/go.mod h1:p2iRAGwDERtqlqzRXnrOVns+ignqQo//hLXqYxZYVNs=
+github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
+github.com/prometheus/client_golang v1.1.0/go.mod h1:I1FGZT9+L76gKKOs5djB6ezCbFQP1xR9D75/vuwEF3g=
+github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
+github.com/prometheus/client_model v0.0.0-20190115171406-56726106282f/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
+github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
+github.com/prometheus/common v0.2.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
+github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
+github.com/prometheus/common v0.6.0/go.mod h1:eBmuwkDJBwy6iBfxCBob6t6dR6ENT/y+J+Zk0j9GMYc=
+github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
+github.com/prometheus/procfs v0.0.0-20190117184657-bf6a532e95b1/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
+github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
+github.com/prometheus/procfs v0.0.3/go.mod h1:4A/X28fw3Fc593LaREMrKMqOKvUAntwMDaekg4FpcdQ=
+github.com/rainycape/memcache v0.0.0-20150622160815-1031fa0ce2f2/go.mod h1:7tZKcyumwBO6qip7RNQ5r77yrssm9bfCowcLEBcU5IA=
+github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
+github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
+github.com/rs/cors v1.7.0 h1:+88SsELBHx5r+hZ8TCkggzSstaWNbDvThkVK8H6f9ik=
+github.com/rs/cors v1.7.0/go.mod h1:gFx+x8UowdsKA9AchylcLynDq+nNFfI8FkUZdN/jGCU=
+github.com/russross/blackfriday/v2 v2.0.1 h1:lPqVAte+HuHNfhJ/0LC98ESWRz8afy9tM/0RK8m9o+Q=
+github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
+github.com/sacloud/libsacloud v1.26.1/go.mod h1:79ZwATmHLIFZIMd7sxA3LwzVy/B77uj3LDoToVTxDoQ=
+github.com/satori/go.uuid v1.2.0/go.mod h1:dA0hQrYB0VpLJoorglMZABFdXlWrHn1NEOzdhQKdks0=
+github.com/shurcooL/sanitized_anchor_name v1.0.0 h1:PdmoCO6wvbs+7yrJyMORt4/BmY5IYyJwS/kOiWx8mHo=
+github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
+github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
+github.com/sirupsen/logrus v1.4.2 h1:SPIRibHv4MatM3XXNO2BJeFLZwZ2LvZgfQ5+UNI2im4=
+github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
+github.com/skratchdot/open-golang v0.0.0-20160302144031-75fb7ed4208c/go.mod h1:sUM3LWHvSMaG192sy56D9F7CNvL7jUJVXoqM1QKLnog=
+github.com/smartystreets/assertions v0.0.0-20180927180507-b2de0cb4f26d/go.mod h1:OnSkiWE9lh6wB0YB77sQom3nweQdgAjqCqsofrRNTgc=
+github.com/smartystreets/goconvey v0.0.0-20190330032615-68dc04aab96a/go.mod h1:syvi0/a8iFYH4r/RixwvyeAJjdLS9QV7WQ/tjFTllLA=
+github.com/spf13/cast v1.3.0 h1:oget//CVOEoFewqQxwr0Ej5yjygnqGkvggSE/gB35Q8=
+github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
+github.com/starlight-go/starlight v0.0.0-20181207205707-b06f321544f3 h1:/fBh1Ot84ILt/ociFHO98wJ9LxIMA3UG8B0unUJPFpY=
+github.com/starlight-go/starlight v0.0.0-20181207205707-b06f321544f3/go.mod h1:pxOc2ZuBV+CNlQgzq/HJ9Z9G/eoEMHFeuGohOvva4Co=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
+github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
+github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
+github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
+github.com/timewasted/linode v0.0.0-20160829202747-37e84520dcf7/go.mod h1:imsgLplxEC/etjIhdr3dNzV3JeT27LbVu5pYWm0JCBY=
+github.com/transip/gotransip v0.0.0-20190812104329-6d8d9179b66f/go.mod h1:i0f4R4o2HM0m3DZYQWsj6/MEowD57VzoH0v3d7igeFY=
+github.com/uber-go/atomic v1.3.2/go.mod h1:/Ct5t2lcmbJ4OSe/waGBoaVvVqtO0bmtfVNex1PFV8g=
+github.com/urfave/cli v1.21.0/go.mod h1:lxDj6qX9Q6lWQxIrbrT0nwecwUtRnhVZAJjJZrVUZZQ=
+github.com/urfave/cli v1.22.1/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
+github.com/vulcand/oxy v1.0.0 h1:7vL5/pjDFzHGbtBEhmlHITUi6KLH4xXTDF33/wrdRKw=
+github.com/vulcand/oxy v1.0.0/go.mod h1:6EXgOAl6CRa46/2ZGcDJKf3ywJUp5WtT7vSlGSkvecI=
+github.com/vulcand/predicate v1.1.0/go.mod h1:mlccC5IRBoc2cIFmCB8ZM62I3VDb6p2GXESMHa3CnZg=
+github.com/vultr/govultr v0.1.4/go.mod h1:9H008Uxr/C4vFNGLqKx232C206GL0PBHzOP0809bGNA=
+github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU=
+github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ=
+github.com/xeipuuv/gojsonschema v1.1.0/go.mod h1:5yf86TLmAcydyeJq5YvxkGPE2fm/u4myDekKRoLuqhs=
+go.opencensus.io v0.20.1/go.mod h1:6WKK9ahsWS3RSO+PY9ZHZUfv2irvY6gN279GOPZjmmk=
+go.opencensus.io v0.20.2/go.mod h1:6WKK9ahsWS3RSO+PY9ZHZUfv2irvY6gN279GOPZjmmk=
+go.opencensus.io v0.21.0/go.mod h1:mSImk1erAIZhrmZN+AvHh14ztQfjbGwt4TtuofqLduU=
+go.starlark.net v0.0.0-20190919145610-979af19b165c h1:WR7X1xgXJlXhQBdorVc9Db3RhwG+J/kp6bLuMyJjfVw=
+go.starlark.net v0.0.0-20190919145610-979af19b165c/go.mod h1:c1/X6cHgvdXj6pUlmWKMkuqRnW4K8x2vwt6JAaaircg=
+go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
+go.uber.org/ratelimit v0.0.0-20180316092928-c15da0234277/go.mod h1:2X8KaoNd1J0lZV+PxJk/5+DGbO/tpwLR1m++a7FnB/Y=
+golang.org/x/crypto v0.0.0-20180621125126-a49355c7e3f8/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
+golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
+golang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
+golang.org/x/crypto v0.0.0-20190228161510-8dd112bcdc25/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
+golang.org/x/crypto v0.0.0-20190418165655-df01cb2cc480/go.mod h1:WFFai1msRO1wXaEeE5yQxYXgSfI8pQAWXbQop6sCtWE=
+golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20190911031432-227b76d455e7/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/crypto v0.0.0-20191010185427-af544f31c8ac h1:/b4NMZurYfBIQyRMqaPGMDeUrSW6gU7/7Hv6owY1Vjk=
+golang.org/x/crypto v0.0.0-20191010185427-af544f31c8ac/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
+golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
+golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
+golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU=
+golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
+golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
+golang.org/x/lint v0.0.0-20190409202823-959b441ac422/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
+golang.org/x/net v0.0.0-20180611182652-db08ff08e862/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190125091013-d26f9f9a57f3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190228165749-92fc7df08ae7/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
+golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190503192946-f4e77d36d62c/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
+golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20190930134127-c5a3c61f89f3/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/net v0.0.0-20191009170851-d66e71096ffb h1:TR699M2v0qoKTOHxeLgp6zPqaQNs74f01a/ob9W0qko=
+golang.org/x/net v0.0.0-20191009170851-d66e71096ffb/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
+golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
+golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
+golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
+golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
+golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
+golang.org/x/sys v0.0.0-20180622082034-63fc586f45fe/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20181122145206-62eef0e2fa9b/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190209173611-3b5209105503/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190228124157-a34e9553db1e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
+golang.org/x/sys v0.0.0-20190403152447-81d4e9dc473e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190507160741-ecd444e8653b/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/sys v0.0.0-20190801041406-cbf593c0f2f3 h1:4y9KwBHBgBNwDbtu44R5o1fdOCQUEXhbk/P4A9WmJq0=
+golang.org/x/sys v0.0.0-20190801041406-cbf593c0f2f3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
+golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
+golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
+golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
+golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/time v0.0.0-20190921001708-c4c64cad1fd0 h1:xQwXv67TxFo9nC1GJFyab5eq/5B590r6RlnL/G8Sz7w=
+golang.org/x/time v0.0.0-20190921001708-c4c64cad1fd0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
+golang.org/x/tools v0.0.0-20180828015842-6cd1fcedba52/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
+golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
+golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
+golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
+google.golang.org/api v0.3.1/go.mod h1:6wY9I6uQWHQ8EM57III9mq/AjF+i8G65rmVagqKMtkk=
+google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
+google.golang.org/api v0.8.0/go.mod h1:o4eAsZoiT+ibD93RtjEohWalFOjRDx6CVaqeizhEnKg=
+google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
+google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
+google.golang.org/appengine v1.5.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
+google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
+google.golang.org/genproto v0.0.0-20180831171423-11092d34479b/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
+google.golang.org/genproto v0.0.0-20190307195333-5fe7a883aa19/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
+google.golang.org/genproto v0.0.0-20190418145605-e7d98fc518a7/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
+google.golang.org/genproto v0.0.0-20190502173448-54afdca5d873/go.mod h1:VzzqZJRnGkLBvHegQrXjBqPurQTc5/KpmUdxsrq26oE=
+google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
+google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
+google.golang.org/grpc v1.19.1/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
+google.golang.org/grpc v1.20.1/go.mod h1:10oTOabMzJvdu6/UiuZezV6QK5dSlG84ov/aaiqXj38=
+gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
+gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/fsnotify.v1 v1.4.7 h1:xOHLXZwVvI9hhs+cLKq5+I5onOuwQLhQwiu63xxlHs4=
+gopkg.in/fsnotify.v1 v1.4.7/go.mod h1:Tz8NjZHkW78fSQdbUxIjBTcgA1z1m8ZHf0WmKUhAMys=
+gopkg.in/h2non/gock.v1 v1.0.15/go.mod h1:sX4zAkdYX1TRGJ2JY156cFspQn4yRWn6p9EMdODlynE=
+gopkg.in/ini.v1 v1.42.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
+gopkg.in/ini.v1 v1.44.0/go.mod h1:pNLf8WUiyNEtQjuu5G5vTm06TEv9tsIgeAvK8hOrP4k=
+gopkg.in/ns1/ns1-go.v2 v2.0.0-20190730140822-b51389932cbc/go.mod h1:VV+3haRsgDiVLxyifmMBrBIuCWFBPYKbRssXB9z67Hw=
+gopkg.in/resty.v1 v1.9.1/go.mod h1:vo52Hzryw9PnPHcJfPsBiFW62XhNx5OczbV9y+IMpgc=
+gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
+gopkg.in/square/go-jose.v2 v2.3.1 h1:SK5KegNXmKmqE342YYN2qPHEnUYeoMiXXl1poUlI+o4=
+gopkg.in/square/go-jose.v2 v2.3.1/go.mod h1:M9dMgbHiYLoDGQrXy7OpJDJWiKiU//h+vD76mk0e1AI=
+gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 h1:uRGJdciOHaEIrze2W8Q3AKkepLTh2hOroT7a+7czfdQ=
+gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7/go.mod h1:dt/ZhP58zS4L8KSrWDmTeBkI65Dw0HsyUHuEVlX15mw=
+gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
+gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
+gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
+honnef.co/go/tools v0.0.0-20180728063816-88497007e858/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
+honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=

listeners.go

@@ -0,0 +1,294 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddy
+
+import (
+	"fmt"
+	"log"
+	"net"
+	"strconv"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"time"
+)
+
+// TODO: Can we use the new UsagePool type?
+
+// Listen returns a listener suitable for use in a Caddy module.
+// Always be sure to close listeners when you are done with them.
+func Listen(network, addr string) (net.Listener, error) {
+	lnKey := network + "/" + addr
+
+	listenersMu.Lock()
+	defer listenersMu.Unlock()
+
+	// if listener already exists, increment usage counter, then return listener
+	if lnUsage, ok := listeners[lnKey]; ok {
+		atomic.AddInt32(&lnUsage.usage, 1)
+		return &fakeCloseListener{usage: &lnUsage.usage, key: lnKey, Listener: lnUsage.ln}, nil
+	}
+
+	// or, create new one and save it
+	ln, err := net.Listen(network, addr)
+	if err != nil {
+		return nil, err
+	}
+
+	// make sure to start its usage counter at 1
+	lnUsage := &listenerUsage{usage: 1, ln: ln}
+	listeners[lnKey] = lnUsage
+
+	return &fakeCloseListener{usage: &lnUsage.usage, key: lnKey, Listener: ln}, nil
+}
+
+// ListenPacket returns a net.PacketConn suitable for use in a Caddy module.
+// Always be sure to close the PacketConn when you are done.
+func ListenPacket(network, addr string) (net.PacketConn, error) {
+	lnKey := network + "/" + addr
+
+	listenersMu.Lock()
+	defer listenersMu.Unlock()
+
+	// if listener already exists, increment usage counter, then return listener
+	if lnUsage, ok := listeners[lnKey]; ok {
+		atomic.AddInt32(&lnUsage.usage, 1)
+		log.Printf("[DEBUG] %s: Usage counter should not go above 2 or maybe 3, is now: %d", lnKey, atomic.LoadInt32(&lnUsage.usage)) // TODO: remove
+		return &fakeClosePacketConn{usage: &lnUsage.usage, key: lnKey, PacketConn: lnUsage.pc}, nil
+	}
+
+	// or, create new one and save it
+	pc, err := net.ListenPacket(network, addr)
+	if err != nil {
+		return nil, err
+	}
+
+	// make sure to start its usage counter at 1
+	lnUsage := &listenerUsage{usage: 1, pc: pc}
+	listeners[lnKey] = lnUsage
+
+	return &fakeClosePacketConn{usage: &lnUsage.usage, key: lnKey, PacketConn: pc}, nil
+}
+
+// fakeCloseListener's Close() method is a no-op. This allows
+// stopping servers that are using the listener without giving
+// up the socket; thus, servers become hot-swappable while the
+// listener remains running. Listeners should be re-wrapped in
+// a new fakeCloseListener each time the listener is reused.
+type fakeCloseListener struct {
+	closed int32  // accessed atomically - TODO: this needs to be shared across the whole app instance, not to cross instance boundaries... hmmm... see #2658 (still relevant?)
+	usage  *int32 // accessed atomically
+	key    string
+	net.Listener
+}
+
+// Accept accepts connections until Close() is called.
+func (fcl *fakeCloseListener) Accept() (net.Conn, error) {
+	// if the listener is already "closed", return error
+	if atomic.LoadInt32(&fcl.closed) == 1 {
+		return nil, fcl.fakeClosedErr()
+	}
+
+	// wrap underlying accept
+	conn, err := fcl.Listener.Accept()
+	if err == nil {
+		return conn, nil
+	}
+
+	if atomic.LoadInt32(&fcl.closed) == 1 {
+		// clear the deadline
+		switch ln := fcl.Listener.(type) {
+		case *net.TCPListener:
+			ln.SetDeadline(time.Time{})
+		case *net.UnixListener:
+			ln.SetDeadline(time.Time{})
+		}
+
+		// if we cancelled the Accept() by setting a deadline
+		// on the listener, we need to make sure any callers of
+		// Accept() think the listener was actually closed;
+		// if we return the timeout error instead, callers might
+		// simply retry, leaking goroutines for longer
+		if netErr, ok := err.(net.Error); ok && netErr.Timeout() {
+			return nil, fcl.fakeClosedErr()
+		}
+	}
+
+	return nil, err
+}
+
+// Close stops accepting new connections without
+// closing the underlying listener, unless no one
+// else is using it.
+func (fcl *fakeCloseListener) Close() error {
+	if atomic.CompareAndSwapInt32(&fcl.closed, 0, 1) {
+		// unfortunately, there is no way to cancel any
+		// currently-blocking calls to Accept() that are
+		// awaiting connections since we're not actually
+		// closing the listener; so we cheat by setting
+		// a deadline in the past, which forces it to
+		// time out; note that this only works for
+		// certain types of listeners...
+		switch ln := fcl.Listener.(type) {
+		case *net.TCPListener:
+			ln.SetDeadline(time.Now().Add(-1 * time.Minute))
+		case *net.UnixListener:
+			ln.SetDeadline(time.Now().Add(-1 * time.Minute))
+		}
+
+		// since we're no longer using this listener,
+		// decrement the usage counter and, if no one
+		// else is using it, close underlying listener
+		if atomic.AddInt32(fcl.usage, -1) == 0 {
+			listenersMu.Lock()
+			delete(listeners, fcl.key)
+			listenersMu.Unlock()
+			err := fcl.Listener.Close()
+			if err != nil {
+				return err
+			}
+		}
+
+	}
+
+	return nil
+}
+
+func (fcl *fakeCloseListener) fakeClosedErr() error {
+	return &net.OpError{
+		Op:   "accept",
+		Net:  fcl.Listener.Addr().Network(),
+		Addr: fcl.Listener.Addr(),
+		Err:  errFakeClosed,
+	}
+}
+
+type fakeClosePacketConn struct {
+	closed int32  // accessed atomically - TODO: this needs to be shared across the whole app instance, not to cross instance boundaries... hmmm... see #2658 (still relevant?)
+	usage  *int32 // accessed atomically
+	key    string
+	net.PacketConn
+}
+
+func (fcpc *fakeClosePacketConn) Close() error {
+	log.Println("[DEBUG] Fake-closing underlying packet conn") // TODO: remove this
+
+	if atomic.CompareAndSwapInt32(&fcpc.closed, 0, 1) {
+		// since we're no longer using this listener,
+		// decrement the usage counter and, if no one
+		// else is using it, close underlying listener
+		if atomic.AddInt32(fcpc.usage, -1) == 0 {
+			listenersMu.Lock()
+			delete(listeners, fcpc.key)
+			listenersMu.Unlock()
+			err := fcpc.PacketConn.Close()
+			if err != nil {
+				return err
+			}
+		}
+	}
+
+	return nil
+}
+
+// ErrFakeClosed is the underlying error value returned by
+// fakeCloseListener.Accept() after Close() has been called,
+// indicating that it is pretending to be closed so that the
+// server using it can terminate, while the underlying
+// socket is actually left open.
+var errFakeClosed = fmt.Errorf("listener 'closed' 😉")
+
+// listenerUsage pairs a net.Listener with a
+// count of how many servers are using it.
+type listenerUsage struct {
+	usage int32 // accessed atomically
+	ln    net.Listener
+	pc    net.PacketConn
+}
+
+var (
+	listeners   = make(map[string]*listenerUsage)
+	listenersMu sync.Mutex
+)
+
+// ParseNetworkAddress parses addr, a string of the form "network/host:port"
+// (with any part optional) into its component parts. Because a port can
+// also be a port range, there may be multiple addresses returned.
+func ParseNetworkAddress(addr string) (network string, addrs []string, err error) {
+	var host, port string
+	network, host, port, err = SplitNetworkAddress(addr)
+	if network == "" {
+		network = "tcp"
+	}
+	if err != nil {
+		return
+	}
+	if network == "unix" || network == "unixgram" || network == "unixpacket" {
+		addrs = []string{host}
+		return
+	}
+	ports := strings.SplitN(port, "-", 2)
+	if len(ports) == 1 {
+		ports = append(ports, ports[0])
+	}
+	var start, end int
+	start, err = strconv.Atoi(ports[0])
+	if err != nil {
+		return
+	}
+	end, err = strconv.Atoi(ports[1])
+	if err != nil {
+		return
+	}
+	if end < start {
+		err = fmt.Errorf("end port must be greater than start port")
+		return
+	}
+	for p := start; p <= end; p++ {
+		addrs = append(addrs, net.JoinHostPort(host, fmt.Sprintf("%d", p)))
+	}
+	return
+}
+
+// SplitNetworkAddress splits a into its network, host, and port components.
+// Note that port may be a port range, or omitted for unix sockets.
+func SplitNetworkAddress(a string) (network, host, port string, err error) {
+	if idx := strings.Index(a, "/"); idx >= 0 {
+		network = strings.ToLower(strings.TrimSpace(a[:idx]))
+		a = a[idx+1:]
+	}
+	if network == "unix" || network == "unixgram" || network == "unixpacket" {
+		host = a
+		return
+	}
+	host, port, err = net.SplitHostPort(a)
+	return
+}
+
+// JoinNetworkAddress combines network, host, and port into a single
+// address string of the form "network/host:port". Port may be a
+// port range. For unix sockets, the network should be "unix" and
+// the path to the socket should be given in the host argument.
+func JoinNetworkAddress(network, host, port string) string {
+	var a string
+	if network != "" {
+		a = network + "/"
+	}
+	a += host
+	if port != "" {
+		a += ":" + port
+	}
+	return a
+}

listeners_test.go

@@ -0,0 +1,221 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddy
+
+import (
+	"reflect"
+	"testing"
+)
+
+func TestSplitNetworkAddress(t *testing.T) {
+	for i, tc := range []struct {
+		input         string
+		expectNetwork string
+		expectHost    string
+		expectPort    string
+		expectErr     bool
+	}{
+		{
+			input:     "",
+			expectErr: true,
+		},
+		{
+			input:     "foo",
+			expectErr: true,
+		},
+		{
+			input:      "foo:1234",
+			expectHost: "foo",
+			expectPort: "1234",
+		},
+		{
+			input:      "foo:1234-5678",
+			expectHost: "foo",
+			expectPort: "1234-5678",
+		},
+		{
+			input:         "udp/foo:1234",
+			expectNetwork: "udp",
+			expectHost:    "foo",
+			expectPort:    "1234",
+		},
+		{
+			input:         "tcp6/foo:1234-5678",
+			expectNetwork: "tcp6",
+			expectHost:    "foo",
+			expectPort:    "1234-5678",
+		},
+		{
+			input:         "udp/",
+			expectNetwork: "udp",
+			expectErr:     true,
+		},
+		{
+			input:         "unix//foo/bar",
+			expectNetwork: "unix",
+			expectHost:    "/foo/bar",
+		},
+		{
+			input:         "unixgram//foo/bar",
+			expectNetwork: "unixgram",
+			expectHost:    "/foo/bar",
+		},
+		{
+			input:         "unixpacket//foo/bar",
+			expectNetwork: "unixpacket",
+			expectHost:    "/foo/bar",
+		},
+	} {
+		actualNetwork, actualHost, actualPort, err := SplitNetworkAddress(tc.input)
+		if tc.expectErr && err == nil {
+			t.Errorf("Test %d: Expected error but got: %v", i, err)
+		}
+		if !tc.expectErr && err != nil {
+			t.Errorf("Test %d: Expected no error but got: %v", i, err)
+		}
+		if actualNetwork != tc.expectNetwork {
+			t.Errorf("Test %d: Expected network '%s' but got '%s'", i, tc.expectNetwork, actualNetwork)
+		}
+		if actualHost != tc.expectHost {
+			t.Errorf("Test %d: Expected host '%s' but got '%s'", i, tc.expectHost, actualHost)
+		}
+		if actualPort != tc.expectPort {
+			t.Errorf("Test %d: Expected port '%s' but got '%s'", i, tc.expectPort, actualPort)
+		}
+	}
+}
+
+func TestJoinNetworkAddress(t *testing.T) {
+	for i, tc := range []struct {
+		network, host, port string
+		expect              string
+	}{
+		{
+			network: "", host: "", port: "",
+			expect: "",
+		},
+		{
+			network: "tcp", host: "", port: "",
+			expect: "tcp/",
+		},
+		{
+			network: "", host: "foo", port: "",
+			expect: "foo",
+		},
+		{
+			network: "", host: "", port: "1234",
+			expect: ":1234",
+		},
+		{
+			network: "", host: "", port: "1234-5678",
+			expect: ":1234-5678",
+		},
+		{
+			network: "", host: "foo", port: "1234",
+			expect: "foo:1234",
+		},
+		{
+			network: "udp", host: "foo", port: "1234",
+			expect: "udp/foo:1234",
+		},
+		{
+			network: "udp", host: "", port: "1234",
+			expect: "udp/:1234",
+		},
+		{
+			network: "unix", host: "/foo/bar", port: "",
+			expect: "unix//foo/bar",
+		},
+	} {
+		actual := JoinNetworkAddress(tc.network, tc.host, tc.port)
+		if actual != tc.expect {
+			t.Errorf("Test %d: Expected '%s' but got '%s'", i, tc.expect, actual)
+		}
+	}
+}
+
+func TestParseNetworkAddress(t *testing.T) {
+	for i, tc := range []struct {
+		input         string
+		expectNetwork string
+		expectAddrs   []string
+		expectErr     bool
+	}{
+		{
+			input:         "",
+			expectNetwork: "tcp",
+			expectErr:     true,
+		},
+		{
+			input:         ":",
+			expectNetwork: "tcp",
+			expectErr:     true,
+		},
+		{
+			input:         ":1234",
+			expectNetwork: "tcp",
+			expectAddrs:   []string{":1234"},
+		},
+		{
+			input:         "tcp/:1234",
+			expectNetwork: "tcp",
+			expectAddrs:   []string{":1234"},
+		},
+		{
+			input:         "tcp6/:1234",
+			expectNetwork: "tcp6",
+			expectAddrs:   []string{":1234"},
+		},
+		{
+			input:         "tcp4/localhost:1234",
+			expectNetwork: "tcp4",
+			expectAddrs:   []string{"localhost:1234"},
+		},
+		{
+			input:         "unix//foo/bar",
+			expectNetwork: "unix",
+			expectAddrs:   []string{"/foo/bar"},
+		},
+		{
+			input:         "localhost:1234-1234",
+			expectNetwork: "tcp",
+			expectAddrs:   []string{"localhost:1234"},
+		},
+		{
+			input:         "localhost:2-1",
+			expectNetwork: "tcp",
+			expectErr:     true,
+		},
+		{
+			input:         "localhost:0",
+			expectNetwork: "tcp",
+			expectAddrs:   []string{"localhost:0"},
+		},
+	} {
+		actualNetwork, actualAddrs, err := ParseNetworkAddress(tc.input)
+		if tc.expectErr && err == nil {
+			t.Errorf("Test %d: Expected error but got: %v", i, err)
+		}
+		if !tc.expectErr && err != nil {
+			t.Errorf("Test %d: Expected no error but got: %v", i, err)
+		}
+		if actualNetwork != tc.expectNetwork {
+			t.Errorf("Test %d: Expected network '%s' but got '%s'", i, tc.expectNetwork, actualNetwork)
+		}
+		if !reflect.DeepEqual(tc.expectAddrs, actualAddrs) {
+			t.Errorf("Test %d: Expected addresses %v but got %v", i, tc.expectAddrs, actualAddrs)
+		}
+	}
+}

main.go

@@ -1,175 +0,0 @@
-package main
-
-import (
-	"bytes"
-	"flag"
-	"fmt"
-	"io/ioutil"
-	"log"
-	"os"
-	"os/exec"
-	"path"
-	"runtime"
-	"strconv"
-	"strings"
-
-	"github.com/mholt/caddy/app"
-	"github.com/mholt/caddy/config"
-	"github.com/mholt/caddy/server"
-)
-
-var (
-	conf    string
-	cpu     string
-	version bool
-)
-
-func init() {
-	flag.StringVar(&conf, "conf", "", "Configuration file to use (default="+config.DefaultConfigFile+")")
-	flag.BoolVar(&app.Http2, "http2", true, "Enable HTTP/2 support") // TODO: temporary flag until http2 merged into std lib
-	flag.BoolVar(&app.Quiet, "quiet", false, "Quiet mode (no initialization output)")
-	flag.StringVar(&cpu, "cpu", "100%", "CPU cap")
-	flag.StringVar(&config.Root, "root", config.DefaultRoot, "Root path to default site")
-	flag.StringVar(&config.Host, "host", config.DefaultHost, "Default host")
-	flag.StringVar(&config.Port, "port", config.DefaultPort, "Default port")
-	flag.BoolVar(&version, "version", false, "Show version")
-}
-
-func main() {
-	flag.Parse()
-
-	if version {
-		fmt.Printf("%s %s\n", app.Name, app.Version)
-		os.Exit(0)
-	}
-
-	// Set CPU cap
-	err := app.SetCPU(cpu)
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	// Load config from file
-	allConfigs, err := loadConfigs()
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	// Group by address (virtual hosts)
-	addresses, err := config.ArrangeBindings(allConfigs)
-	if err != nil {
-		log.Fatal(err)
-	}
-
-	// Start each server with its one or more configurations
-	for addr, configs := range addresses {
-		s, err := server.New(addr.String(), configs, configs[0].TLS.Enabled)
-		if err != nil {
-			log.Fatal(err)
-		}
-		s.HTTP2 = app.Http2 // TODO: This setting is temporary
-		app.Wg.Add(1)
-		go func(s *server.Server) {
-			defer app.Wg.Done()
-			err := s.Serve()
-			if err != nil {
-				log.Fatal(err) // kill whole process to avoid a half-alive zombie server
-			}
-		}(s)
-
-		app.Servers = append(app.Servers, s)
-	}
-
-	// Show initialization output
-	if !app.Quiet {
-		var checkedFdLimit bool
-		for addr, configs := range addresses {
-			for _, conf := range configs {
-				// Print address of site
-				fmt.Println(conf.Address())
-
-				// Note if non-localhost site resolves to loopback interface
-				if addr.IP.IsLoopback() && !isLocalhost(conf.Host) {
-					fmt.Printf("Notice: %s is only accessible on this machine (%s)\n",
-						conf.Host, addr.IP.String())
-				}
-			}
-
-			if !checkedFdLimit && !addr.IP.IsLoopback() {
-				checkFdlimit()
-				checkedFdLimit = true
-			}
-		}
-	}
-
-	// Wait for all listeners to stop
-	app.Wg.Wait()
-}
-
-// checkFdlimit issues a warning if the OS max file descriptors is below a recommended minimum.
-func checkFdlimit() {
-	const min = 4096
-
-	// Warn if ulimit is too low for production sites
-	if runtime.GOOS == "linux" || runtime.GOOS == "darwin" {
-		out, err := exec.Command("sh", "-c", "ulimit -n").Output() // use sh because ulimit isn't in Linux $PATH
-		if err == nil {
-			// Note that an error here need not be reported
-			lim, err := strconv.Atoi(string(bytes.TrimSpace(out)))
-			if err == nil && lim < min {
-				fmt.Printf("Warning: File descriptor limit %d is too low for production sites.\nAt least %d is recommended. Set with \"ulimit -n %d\".\n", lim, min, min)
-			}
-		}
-	}
-}
-
-// isLocalhost returns true if the string looks explicitly like a localhost address.
-func isLocalhost(s string) bool {
-	return s == "localhost" || s == "::1" || strings.HasPrefix(s, "127.")
-}
-
-// loadConfigs loads configuration from a file or stdin (piped).
-// Configuration is obtained from one of three sources, tried
-// in this order: 1. -conf flag, 2. stdin, 3. Caddyfile.
-// If none of those are available, a default configuration is
-// loaded.
-func loadConfigs() ([]server.Config, error) {
-	// -conf flag
-	if conf != "" {
-		file, err := os.Open(conf)
-		if err != nil {
-			return []server.Config{}, err
-		}
-		defer file.Close()
-		return config.Load(path.Base(conf), file)
-	}
-
-	// stdin
-	fi, err := os.Stdin.Stat()
-	if err == nil && fi.Mode()&os.ModeCharDevice == 0 {
-		// Note that a non-nil error is not a problem. Windows
-		// will not create a stdin if there is no pipe, which
-		// produces an error when calling Stat(). But Unix will
-		// make one either way, which is why we also check that
-		// bitmask.
-		confBody, err := ioutil.ReadAll(os.Stdin)
-		if err != nil {
-			log.Fatal(err)
-		}
-		if len(confBody) > 0 {
-			return config.Load("stdin", bytes.NewReader(confBody))
-		}
-	}
-
-	// Caddyfile
-	file, err := os.Open(config.DefaultConfigFile)
-	if err != nil {
-		if os.IsNotExist(err) {
-			return []server.Config{config.Default()}, nil
-		}
-		return []server.Config{}, err
-	}
-	defer file.Close()
-
-	return config.Load(config.DefaultConfigFile, file)
-}

middleware/basicauth/basicauth.go

@@ -1,65 +0,0 @@
-// Package basicauth implements HTTP Basic Authentication.
-package basicauth
-
-import (
-	"net/http"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-// BasicAuth is middleware to protect resources with a username and password.
-// Note that HTTP Basic Authentication is not secure by itself and should
-// not be used to protect important assets without HTTPS. Even then, the
-// security of HTTP Basic Auth is disputed. Use discretion when deciding
-// what to protect with BasicAuth.
-type BasicAuth struct {
-	Next  middleware.Handler
-	Rules []Rule
-}
-
-// ServeHTTP implements the middleware.Handler interface.
-func (a BasicAuth) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-
-	var hasAuth bool
-	var isAuthenticated bool
-
-	for _, rule := range a.Rules {
-		for _, res := range rule.Resources {
-			if !middleware.Path(r.URL.Path).Matches(res) {
-				continue
-			}
-
-			// Path matches; parse auth header
-			username, password, ok := r.BasicAuth()
-			hasAuth = true
-
-			// Check credentials
-			if !ok || username != rule.Username || password != rule.Password {
-				continue
-			}
-			// flag set only on success authentication
-			isAuthenticated = true
-		}
-	}
-
-	if hasAuth {
-		if !isAuthenticated {
-			w.Header().Set("WWW-Authenticate", "Basic")
-			return http.StatusUnauthorized, nil
-		}
-		// "It's an older code, sir, but it checks out. I was about to clear them."
-		return a.Next.ServeHTTP(w, r)
-	}
-
-	// Pass-thru when no paths match
-	return a.Next.ServeHTTP(w, r)
-}
-
-// Rule represents a BasicAuth rule. A username and password
-// combination protect the associated resources, which are
-// file or directory paths.
-type Rule struct {
-	Username  string
-	Password  string
-	Resources []string
-}

middleware/basicauth/basicauth_test.go

@@ -1,113 +0,0 @@
-package basicauth
-
-import (
-	"encoding/base64"
-	"fmt"
-	"net/http"
-	"net/http/httptest"
-	"testing"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-func TestBasicAuth(t *testing.T) {
-
-	rw := BasicAuth{
-		Next: middleware.HandlerFunc(contentHandler),
-		Rules: []Rule{
-			{Username: "test", Password: "ttest", Resources: []string{"/testing"}},
-		},
-	}
-
-	tests := []struct {
-		from   string
-		result int
-		cred   string
-	}{
-		{"/testing", http.StatusUnauthorized, "ttest:test"},
-		{"/testing", http.StatusOK, "test:ttest"},
-		{"/testing", http.StatusUnauthorized, ""},
-	}
-
-	for i, test := range tests {
-
-		req, err := http.NewRequest("GET", test.from, nil)
-		if err != nil {
-			t.Fatalf("Test %d: Could not create HTTP request %v", i, err)
-		}
-		auth := "Basic " + base64.StdEncoding.EncodeToString([]byte(test.cred))
-		req.Header.Set("Authorization", auth)
-
-		rec := httptest.NewRecorder()
-		result, err := rw.ServeHTTP(rec, req)
-		if err != nil {
-			t.Fatalf("Test %d: Could not ServeHTTP %v", i, err)
-		}
-		if result != test.result {
-			t.Errorf("Test %d: Expected Header '%d' but was '%d'",
-				i, test.result, result)
-		}
-		if result == http.StatusUnauthorized {
-			headers := rec.Header()
-			if val, ok := headers["Www-Authenticate"]; ok {
-				if val[0] != "Basic" {
-					t.Errorf("Test %d, Www-Authenticate should be %s provided %s", i, "Basic", val[0])
-				}
-			} else {
-				t.Errorf("Test %d, should provide a header Www-Authenticate", i)
-			}
-		}
-
-	}
-
-}
-
-func TestMultipleOverlappingRules(t *testing.T) {
-	rw := BasicAuth{
-		Next: middleware.HandlerFunc(contentHandler),
-		Rules: []Rule{
-			{Username: "t", Password: "p1", Resources: []string{"/t"}},
-			{Username: "t1", Password: "p2", Resources: []string{"/t/t"}},
-		},
-	}
-
-	tests := []struct {
-		from   string
-		result int
-		cred   string
-	}{
-		{"/t", http.StatusOK, "t:p1"},
-		{"/t/t", http.StatusOK, "t:p1"},
-		{"/t/t", http.StatusOK, "t1:p2"},
-		{"/a", http.StatusOK, "t1:p2"},
-		{"/t/t", http.StatusUnauthorized, "t1:p3"},
-		{"/t", http.StatusUnauthorized, "t1:p2"},
-	}
-
-	for i, test := range tests {
-
-		req, err := http.NewRequest("GET", test.from, nil)
-		if err != nil {
-			t.Fatalf("Test %d: Could not create HTTP request %v", i, err)
-		}
-		auth := "Basic " + base64.StdEncoding.EncodeToString([]byte(test.cred))
-		req.Header.Set("Authorization", auth)
-
-		rec := httptest.NewRecorder()
-		result, err := rw.ServeHTTP(rec, req)
-		if err != nil {
-			t.Fatalf("Test %d: Could not ServeHTTP %v", i, err)
-		}
-		if result != test.result {
-			t.Errorf("Test %d: Expected Header '%d' but was '%d'",
-				i, test.result, result)
-		}
-
-	}
-
-}
-
-func contentHandler(w http.ResponseWriter, r *http.Request) (int, error) {
-	fmt.Fprintf(w, r.URL.String())
-	return http.StatusOK, nil
-}

middleware/browse/browse.go

@@ -1,183 +0,0 @@
-// Package browse provides middleware for listing files in a directory
-// when directory path is requested instead of a specific file.
-package browse
-
-import (
-	"bytes"
-	"html/template"
-	"net/http"
-	"net/url"
-	"os"
-	"path"
-	"strings"
-	"time"
-
-	"github.com/dustin/go-humanize"
-	"github.com/mholt/caddy/middleware"
-)
-
-// Browse is an http.Handler that can show a file listing when
-// directories in the given paths are specified.
-type Browse struct {
-	Next    middleware.Handler
-	Root    string
-	Configs []Config
-}
-
-// Config is a configuration for browsing in a particular path.
-type Config struct {
-	PathScope string
-	Template  *template.Template
-}
-
-// A Listing is used to fill out a template.
-type Listing struct {
-	// The name of the directory (the last element of the path)
-	Name string
-
-	// The full path of the request
-	Path string
-
-	// Whether the parent directory is browsable
-	CanGoUp bool
-
-	// The items (files and folders) in the path
-	Items []FileInfo
-}
-
-// FileInfo is the info about a particular file or directory
-type FileInfo struct {
-	IsDir   bool
-	Name    string
-	Size    int64
-	URL     string
-	ModTime time.Time
-	Mode    os.FileMode
-}
-
-// HumanSize returns the size of the file as a human-readable string.
-func (fi FileInfo) HumanSize() string {
-	return humanize.Bytes(uint64(fi.Size))
-}
-
-// HumanModTime returns the modified time of the file as a human-readable string.
-func (fi FileInfo) HumanModTime(format string) string {
-	return fi.ModTime.Format(format)
-}
-
-var IndexPages = []string{
-	"index.html",
-	"index.htm",
-	"default.html",
-	"default.htm",
-}
-
-// ServeHTTP implements the middleware.Handler interface.
-func (b Browse) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-	filename := b.Root + r.URL.Path
-
-	info, err := os.Stat(filename)
-	if err != nil {
-		return b.Next.ServeHTTP(w, r)
-	}
-
-	if !info.IsDir() {
-		return b.Next.ServeHTTP(w, r)
-	}
-
-	// See if there's a browse configuration to match the path
-	for _, bc := range b.Configs {
-		if !middleware.Path(r.URL.Path).Matches(bc.PathScope) {
-			continue
-		}
-
-		// Browsing navigation gets messed up if browsing a directory
-		// that doesn't end in "/" (which it should, anyway)
-		if r.URL.Path[len(r.URL.Path)-1] != '/' {
-			http.Redirect(w, r, r.URL.Path+"/", http.StatusTemporaryRedirect)
-			return 0, nil
-		}
-
-		// Load directory contents
-		file, err := os.Open(b.Root + r.URL.Path)
-		if err != nil {
-			if os.IsPermission(err) {
-				return http.StatusForbidden, err
-			}
-			return http.StatusNotFound, err
-		}
-		defer file.Close()
-
-		files, err := file.Readdir(-1)
-		if err != nil {
-			return http.StatusForbidden, err
-		}
-
-		// Assemble listing of directory contents
-		var fileinfos []FileInfo
-		var abort bool // we bail early if we find an index file
-		for _, f := range files {
-			name := f.Name()
-
-			// Directory is not browseable if it contains index file
-			for _, indexName := range IndexPages {
-				if name == indexName {
-					abort = true
-					break
-				}
-			}
-			if abort {
-				break
-			}
-
-			if f.IsDir() {
-				name += "/"
-			}
-			url := url.URL{Path: name}
-
-			fileinfos = append(fileinfos, FileInfo{
-				IsDir:   f.IsDir(),
-				Name:    f.Name(),
-				Size:    f.Size(),
-				URL:     url.String(),
-				ModTime: f.ModTime(),
-				Mode:    f.Mode(),
-			})
-		}
-		if abort {
-			// this dir has an index file, so not browsable
-			continue
-		}
-
-		// Determine if user can browse up another folder
-		var canGoUp bool
-		curPath := strings.TrimSuffix(r.URL.Path, "/")
-		for _, other := range b.Configs {
-			if strings.HasPrefix(path.Dir(curPath), other.PathScope) {
-				canGoUp = true
-				break
-			}
-		}
-
-		listing := Listing{
-			Name:    path.Base(r.URL.Path),
-			Path:    r.URL.Path,
-			CanGoUp: canGoUp,
-			Items:   fileinfos,
-		}
-
-		var buf bytes.Buffer
-		err = bc.Template.Execute(&buf, listing)
-		if err != nil {
-			return http.StatusInternalServerError, err
-		}
-
-		w.Header().Set("Content-Type", "text/html; charset=utf-8")
-		buf.WriteTo(w)
-
-		return http.StatusOK, nil
-	}
-
-	// Didn't qualify; pass-thru
-	return b.Next.ServeHTTP(w, r)
-}

middleware/commands.go

@@ -1,27 +0,0 @@
-package middleware
-
-import (
-	"errors"
-
-	"github.com/flynn/go-shlex"
-)
-
-// SplitCommandAndArgs takes a command string and parses it
-// shell-style into the command and its separate arguments.
-func SplitCommandAndArgs(command string) (cmd string, args []string, err error) {
-	parts, err := shlex.Split(command)
-	if err != nil {
-		err = errors.New("Error parsing command: " + err.Error())
-		return
-	} else if len(parts) == 0 {
-		err = errors.New("No command contained in '" + command + "'")
-		return
-	}
-
-	cmd = parts[0]
-	if len(parts) > 1 {
-		args = parts[1:]
-	}
-
-	return
-}

middleware/errors/errors.go

@@ -1,114 +0,0 @@
-// Package errors implements an HTTP error handling middleware.
-package errors
-
-import (
-	"fmt"
-	"io"
-	"log"
-	"net/http"
-	"os"
-	"runtime"
-	"strings"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-// ErrorHandler handles HTTP errors (or errors from other middleware).
-type ErrorHandler struct {
-	Next       middleware.Handler
-	ErrorPages map[int]string // map of status code to filename
-	LogFile    string
-	Log        *log.Logger
-}
-
-func (h ErrorHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-	defer h.recovery(w, r)
-
-	status, err := h.Next.ServeHTTP(w, r)
-
-	if err != nil {
-		h.Log.Printf("[ERROR %d %s] %v", status, r.URL.Path, err)
-	}
-
-	if status >= 400 {
-		h.errorPage(w, status)
-		return 0, err // status < 400 signals that a response has been written
-	}
-
-	return status, err
-}
-
-// errorPage serves a static error page to w according to the status
-// code. If there is an error serving the error page, a plaintext error
-// message is written instead, and the extra error is logged.
-func (h ErrorHandler) errorPage(w http.ResponseWriter, code int) {
-	defaultBody := fmt.Sprintf("%d %s", code, http.StatusText(code))
-
-	// See if an error page for this status code was specified
-	if pagePath, ok := h.ErrorPages[code]; ok {
-
-		// Try to open it
-		errorPage, err := os.Open(pagePath)
-		if err != nil {
-			// An error handling an error... <insert grumpy cat here>
-			h.Log.Printf("HTTP %d could not load error page %s: %v", code, pagePath, err)
-			http.Error(w, defaultBody, code)
-			return
-		}
-		defer errorPage.Close()
-
-		// Copy the page body into the response
-		w.Header().Set("Content-Type", "text/html; charset=utf-8")
-		w.WriteHeader(code)
-		_, err = io.Copy(w, errorPage)
-
-		if err != nil {
-			// Epic fail... sigh.
-			h.Log.Printf("HTTP %d could not respond with %s: %v", code, pagePath, err)
-			http.Error(w, defaultBody, code)
-		}
-
-		return
-	}
-
-	// Default error response
-	http.Error(w, defaultBody, code)
-}
-
-func (h ErrorHandler) recovery(w http.ResponseWriter, r *http.Request) {
-	rec := recover()
-	if rec == nil {
-		return
-	}
-
-	// Obtain source of panic
-	// From: https://gist.github.com/swdunlop/9629168
-	var name, file string // function name, file name
-	var line int
-	var pc [16]uintptr
-	n := runtime.Callers(3, pc[:])
-	for _, pc := range pc[:n] {
-		fn := runtime.FuncForPC(pc)
-		if fn == nil {
-			continue
-		}
-		file, line = fn.FileLine(pc)
-		name = fn.Name()
-		if !strings.HasPrefix(name, "runtime.") {
-			break
-		}
-	}
-
-	// Trim file path
-	delim := "/caddy/"
-	pkgPathPos := strings.Index(file, delim)
-	if pkgPathPos > -1 && len(file) > pkgPathPos+len(delim) {
-		file = file[pkgPathPos+len(delim):]
-	}
-
-	// Currently we don't use the function name, as file:line is more conventional
-	h.Log.Printf("[PANIC %s] %s:%d - %v", r.URL.String(), file, line, rec)
-	h.errorPage(w, http.StatusInternalServerError)
-}
-
-const DefaultLogFilename = "error.log"

middleware/extensions/ext.go

@@ -1,52 +0,0 @@
-// Package extensions contains middleware for clean URLs.
-//
-// The root path of the site is passed in as well as possible extensions
-// to try internally for paths requested that don't match an existing
-// resource. The first path+ext combination that matches a valid file
-// will be used.
-package extensions
-
-import (
-	"net/http"
-	"os"
-	"path"
-	"strings"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-// Ext can assume an extension from clean URLs.
-// It tries extensions in the order listed in Extensions.
-type Ext struct {
-	// Next handler in the chain
-	Next middleware.Handler
-
-	// Path to ther root of the site
-	Root string
-
-	// List of extensions to try
-	Extensions []string
-}
-
-// ServeHTTP implements the middleware.Handler interface.
-func (e Ext) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-	urlpath := strings.TrimSuffix(r.URL.Path, "/")
-	if path.Ext(urlpath) == "" {
-		for _, ext := range e.Extensions {
-			if resourceExists(e.Root, urlpath+ext) {
-				r.URL.Path = urlpath + ext
-				break
-			}
-		}
-	}
-	return e.Next.ServeHTTP(w, r)
-}
-
-// resourceExists returns true if the file specified at
-// root + path exists; false otherwise.
-func resourceExists(root, path string) bool {
-	_, err := os.Stat(root + path)
-	// technically we should use os.IsNotExist(err)
-	// but we don't handle any other kinds of errors anyway
-	return err == nil
-}

middleware/fastcgi/fastcgi.go

@@ -1,238 +0,0 @@
-// Package fastcgi has middleware that acts as a FastCGI client. Requests
-// that get forwarded to FastCGI stop the middleware execution chain.
-// The most common use for this package is to serve PHP websites via php-fpm.
-package fastcgi
-
-import (
-	"io"
-	"net/http"
-	"os"
-	"path/filepath"
-	"strconv"
-	"strings"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-// Handler is a middleware type that can handle requests as a FastCGI client.
-type Handler struct {
-	Next    middleware.Handler
-	Rules   []Rule
-	Root    string
-	AbsRoot string // same as root, but absolute path
-	FileSys http.FileSystem
-
-	// These are sent to CGI scripts in env variables
-	SoftwareName    string
-	SoftwareVersion string
-	ServerName      string
-	ServerPort      string
-}
-
-// ServeHTTP satisfies the middleware.Handler interface.
-func (h Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-	for _, rule := range h.Rules {
-
-		// First requirement: Base path must match
-		if !middleware.Path(r.URL.Path).Matches(rule.Path) {
-			continue
-		}
-
-		// In addition to matching the path, a request must meet some
-		// other criteria before being proxied as FastCGI. For example,
-		// we probably want to exclude static assets (CSS, JS, images...)
-		// but we also want to be flexible for the script we proxy to.
-
-		fpath := r.URL.Path
-		if idx, ok := middleware.IndexFile(h.FileSys, fpath, rule.IndexFiles); ok {
-			fpath = idx
-		}
-
-		// These criteria work well in this order for PHP sites
-		if fpath[len(fpath)-1] == '/' || strings.HasSuffix(fpath, rule.Ext) || !h.exists(fpath) {
-
-			// Create environment for CGI script
-			env, err := h.buildEnv(r, rule, fpath)
-			if err != nil {
-				return http.StatusInternalServerError, err
-			}
-
-			// Connect to FastCGI gateway
-			var fcgi *FCGIClient
-
-			// check if unix socket or tcp
-			if strings.HasPrefix(rule.Address, "/") {
-				fcgi, err = Dial("unix", rule.Address)
-			} else {
-				fcgi, err = Dial("tcp", rule.Address)
-			}
-			if err != nil {
-				return http.StatusBadGateway, err
-			}
-
-			var resp *http.Response
-			contentLength, _ := strconv.Atoi(r.Header.Get("Content-Length"))
-			switch r.Method {
-			case "HEAD":
-				resp, err = fcgi.Head(env)
-			case "GET":
-				resp, err = fcgi.Get(env)
-			case "OPTIONS":
-				resp, err = fcgi.Options(env)
-			case "POST":
-				resp, err = fcgi.Post(env, r.Header.Get("Content-Type"), r.Body, contentLength)
-			case "PUT":
-				resp, err = fcgi.Put(env, r.Header.Get("Content-Type"), r.Body, contentLength)
-			case "PATCH":
-				resp, err = fcgi.Patch(env, r.Header.Get("Content-Type"), r.Body, contentLength)
-			case "DELETE":
-				resp, err = fcgi.Delete(env, r.Header.Get("Content-Type"), r.Body, contentLength)
-			default:
-				return http.StatusMethodNotAllowed, nil
-			}
-
-			if resp.Body != nil {
-				defer resp.Body.Close()
-			}
-
-			if err != nil && err != io.EOF {
-				return http.StatusBadGateway, err
-			}
-
-			// Write the response header
-			for key, vals := range resp.Header {
-				for _, val := range vals {
-					w.Header().Add(key, val)
-				}
-			}
-			w.WriteHeader(resp.StatusCode)
-
-			// Write the response body
-			// TODO: If this has an error, the response will already be
-			// partly written. We should copy out of resp.Body into a buffer
-			// first, then write it to the response...
-			_, err = io.Copy(w, resp.Body)
-			if err != nil {
-				return http.StatusBadGateway, err
-			}
-
-			return 0, nil
-		}
-	}
-
-	return h.Next.ServeHTTP(w, r)
-}
-
-func (h Handler) exists(path string) bool {
-	if _, err := os.Stat(h.Root + path); err == nil {
-		return true
-	}
-	return false
-}
-
-// buildEnv returns a set of CGI environment variables for the request.
-func (h Handler) buildEnv(r *http.Request, rule Rule, fpath string) (map[string]string, error) {
-	var env map[string]string
-
-	// Get absolute path of requested resource
-	absPath := filepath.Join(h.AbsRoot, fpath)
-
-	// Separate remote IP and port; more lenient than net.SplitHostPort
-	var ip, port string
-	if idx := strings.Index(r.RemoteAddr, ":"); idx > -1 {
-		ip = r.RemoteAddr[:idx]
-		port = r.RemoteAddr[idx+1:]
-	} else {
-		ip = r.RemoteAddr
-	}
-
-	// Split path in preparation for env variables
-	splitPos := strings.Index(fpath, rule.SplitPath)
-	var docURI, scriptName, scriptFilename, pathInfo string
-	if splitPos == -1 {
-		// Request doesn't have the extension, so assume index file in root
-		docURI = "/" + rule.IndexFiles[0]
-		scriptName = "/" + rule.IndexFiles[0]
-		scriptFilename = filepath.Join(h.AbsRoot, rule.IndexFiles[0])
-		pathInfo = fpath
-	} else {
-		// Request has the extension; path was split successfully
-		docURI = fpath[:splitPos+len(rule.SplitPath)]
-		pathInfo = fpath[splitPos+len(rule.SplitPath):]
-		scriptName = fpath
-		scriptFilename = absPath
-	}
-
-	// Some variables are unused but cleared explicitly to prevent
-	// the parent environment from interfering.
-	env = map[string]string{
-
-		// Variables defined in CGI 1.1 spec
-		"AUTH_TYPE":         "", // Not used
-		"CONTENT_LENGTH":    r.Header.Get("Content-Length"),
-		"CONTENT_TYPE":      r.Header.Get("Content-Type"),
-		"GATEWAY_INTERFACE": "CGI/1.1",
-		"PATH_INFO":         pathInfo,
-		"PATH_TRANSLATED":   filepath.Join(h.AbsRoot, pathInfo), // Info: http://www.oreilly.com/openbook/cgi/ch02_04.html
-		"QUERY_STRING":      r.URL.RawQuery,
-		"REMOTE_ADDR":       ip,
-		"REMOTE_HOST":       ip, // For speed, remote host lookups disabled
-		"REMOTE_PORT":       port,
-		"REMOTE_IDENT":      "", // Not used
-		"REMOTE_USER":       "", // Not used
-		"REQUEST_METHOD":    r.Method,
-		"SERVER_NAME":       h.ServerName,
-		"SERVER_PORT":       h.ServerPort,
-		"SERVER_PROTOCOL":   r.Proto,
-		"SERVER_SOFTWARE":   h.SoftwareName + "/" + h.SoftwareVersion,
-
-		// Other variables
-		"DOCUMENT_ROOT":   h.AbsRoot,
-		"DOCUMENT_URI":    docURI,
-		"HTTP_HOST":       r.Host, // added here, since not always part of headers
-		"REQUEST_URI":     r.URL.RequestURI(),
-		"SCRIPT_FILENAME": scriptFilename,
-		"SCRIPT_NAME":     scriptName,
-	}
-
-	// Add env variables from config
-	for _, envVar := range rule.EnvVars {
-		env[envVar[0]] = envVar[1]
-	}
-
-	// Add all HTTP headers to env variables
-	for field, val := range r.Header {
-		header := strings.ToUpper(field)
-		header = headerNameReplacer.Replace(header)
-		env["HTTP_"+header] = strings.Join(val, ", ")
-	}
-
-	return env, nil
-}
-
-// Rule represents a FastCGI handling rule.
-type Rule struct {
-	// The base path to match. Required.
-	Path string
-
-	// The address of the FastCGI server. Required.
-	Address string
-
-	// Always process files with this extension with fastcgi.
-	Ext string
-
-	// The path in the URL will be split into two, with the first piece ending
-	// with the value of SplitPath. The first piece will be assumed as the
-	// actual resource (CGI script) name, and the second piece will be set to
-	// PATH_INFO for the CGI script to use.
-	SplitPath string
-
-	// If the URL ends with '/' (which indicates a directory), these index
-	// files will be tried instead.
-	IndexFiles []string
-
-	// Environment Variables
-	EnvVars [][2]string
-}
-
-var headerNameReplacer = strings.NewReplacer(" ", "_", "-", "_")

middleware/fastcgi/fcgi_test.php

@@ -1,79 +0,0 @@
-<?php
-
-ini_set("display_errors",1);
-
-echo "resp: start\n";//.print_r($GLOBALS,1)."\n".print_r($_SERVER,1)."\n";
-
-//echo print_r($_SERVER,1)."\n";
-
-$length = 0;
-$stat = "PASSED";
-
-$ret = "[";
-
-if (count($_POST) || count($_FILES)) {
-	foreach($_POST as $key => $val) {
-		$md5 = md5($val);
-		
-		if ($key != $md5) {
-			$stat = "FAILED";
-			echo "server:err ".$md5." != ".$key."\n";
-		}
-		
-		$length += strlen($key) + strlen($val);
-		
-		$ret .= $key."(".strlen($key).") ";
-	}
-	$ret .= "] [";
-	foreach ($_FILES as $k0 => $val) {
-
-		$error = $val["error"];
-		if ($error == UPLOAD_ERR_OK) {
-			$tmp_name = $val["tmp_name"];
-			$name = $val["name"];
-			$datafile = "/tmp/test.go";
-			move_uploaded_file($tmp_name, $datafile);
-			$md5 = md5_file($datafile);
-
-			if ($k0 != $md5) {
-				$stat = "FAILED";
-				echo "server:err ".$md5." != ".$key."\n";
-			}      
-			
-			$length += strlen($k0) + filesize($datafile);
-			
-			unlink($datafile);
-			$ret .= $k0."(".strlen($k0).") ";
-		}
-		else{
-			$stat = "FAILED";
-			echo "server:file err ".file_upload_error_message($error)."\n";
-		}
-	}
-	$ret .= "]";
-	echo "server:got data length " .$length."\n";
-}
-
-
-echo "-{$stat}-POST(".count($_POST).") FILE(".count($_FILES).")\n";
-
-function file_upload_error_message($error_code) {
-	switch ($error_code) { 
-		case UPLOAD_ERR_INI_SIZE: 
-		return 'The uploaded file exceeds the upload_max_filesize directive in php.ini'; 
-		case UPLOAD_ERR_FORM_SIZE: 
-		return 'The uploaded file exceeds the MAX_FILE_SIZE directive that was specified in the HTML form'; 
-		case UPLOAD_ERR_PARTIAL: 
-		return 'The uploaded file was only partially uploaded'; 
-		case UPLOAD_ERR_NO_FILE: 
-		return 'No file was uploaded'; 
-		case UPLOAD_ERR_NO_TMP_DIR: 
-		return 'Missing a temporary folder'; 
-		case UPLOAD_ERR_CANT_WRITE: 
-		return 'Failed to write file to disk'; 
-		case UPLOAD_ERR_EXTENSION: 
-		return 'File upload stopped by extension'; 
-		default: 
-		return 'Unknown upload error'; 
-	} 
-} 

middleware/git/doc.go

@@ -1,71 +0,0 @@
-// Package git is the middleware that pull sites from git repo
-//
-// Caddyfile Syntax :
-//	git repo path {
-//		repo
-//		path
-//		branch
-//		key
-//		interval
-//		then command args
-//	}
-//	repo 	- git repository
-// 		compulsory. Both ssh (e.g. git@github.com:user/project.git)
-// 		and https(e.g. https://github.com/user/project) are supported.
-//		Can be specified in either config block or top level
-//
-// 	path 	- directory to pull into, relative to site root
-//		optional. Defaults to site root.
-//
-// 	branch 	- git branch or tag
-//		optional. Defaults to master
-//
-// 	key 	- path to private ssh key
-//		optional. Required for private repositories. e.g. /home/user/.ssh/id_rsa
-//
-// 	interval- interval between git pulls in seconds
-//		optional. Defaults to 3600 (1 Hour).
-//
-//	then	- command to execute after successful pull
-//		optional. If set, will execute only when there are new changes.
-//
-// Examples :
-//
-// public repo pulled into site root
-//	git github.com/user/myproject
-//
-// public repo pulled into <root>/mysite
-//	git https://github.com/user/myproject mysite
-//
-// private repo pulled into <root>/mysite with tag v1.0 and interval of 1 day.
-//	git {
-//		repo 	git@github.com:user/myproject
-//		branch 	v1.0
-//		path	mysite
-//		key 	/home/user/.ssh/id_rsa
-//		interval 86400 # 1 day
-//	}
-//
-// Caddyfile with private git repo and php support via fastcgi.
-// path defaults to /var/www/html/myphpsite as specified in root config.
-//
-//	0.0.0.0:8080
-//
-//	git {
-//		repo 	git@github.com:user/myphpsite
-//		key 	/home/user/.ssh/id_rsa
-//		interval 86400 # 1 day
-//	}
-//
-//	fastcgi / 127.0.0.1:9000 php
-//
-//	root /var/www/html/myphpsite
-//
-// A pull is first attempted after initialization. Afterwards, a pull is attempted
-// after request to server and if time taken since last successful pull is higher than interval.
-//
-// After the first successful pull (should be during initialization except an error occurs),
-// subsequent pulls are done in background and do not impact request time.
-//
-// Note: private repositories are currently only supported and tested on Linux and OSX
-package git

middleware/git/git.go

@@ -1,360 +0,0 @@
-package git
-
-import (
-	"bytes"
-	"fmt"
-	"log"
-	"os"
-	"strings"
-	"sync"
-	"time"
-
-	"github.com/mholt/caddy/middleware"
-	"github.com/mholt/caddy/middleware/git/gitos"
-)
-
-// DefaultInterval is the minimum interval to delay before
-// requesting another git pull
-const DefaultInterval time.Duration = time.Hour * 1
-
-// Number of retries if git pull fails
-const numRetries = 3
-
-// gitBinary holds the absolute path to git executable
-var gitBinary string
-
-// shell holds the shell to be used. Either sh or bash.
-var shell string
-
-// initMutex prevents parallel attempt to validate
-// git requirements.
-var initMutex = sync.Mutex{}
-
-// Logger is used to log errors; if nil, the default log.Logger is used.
-var Logger *log.Logger
-
-// logger is an helper function to retrieve the available logger
-func logger() *log.Logger {
-	if Logger == nil {
-		Logger = log.New(os.Stderr, "", log.LstdFlags)
-	}
-	return Logger
-}
-
-// Repo is the structure that holds required information
-// of a git repository.
-type Repo struct {
-	URL        string        // Repository URL
-	Path       string        // Directory to pull to
-	Host       string        // Git domain host e.g. github.com
-	Branch     string        // Git branch
-	KeyPath    string        // Path to private ssh key
-	Interval   time.Duration // Interval between pulls
-	Then       string        // Command to execute after successful git pull
-	pulled     bool          // true if there was a successful pull
-	lastPull   time.Time     // time of the last successful pull
-	lastCommit string        // hash for the most recent commit
-	sync.Mutex
-}
-
-// Pull attempts a git clone.
-// It retries at most numRetries times if error occurs
-func (r *Repo) Pull() error {
-	r.Lock()
-	defer r.Unlock()
-
-	// prevent a pull if the last one was less than 5 seconds ago
-	if time.Since(r.lastPull) < 5*time.Second {
-		return nil
-	}
-
-	// keep last commit hash for comparison later
-	lastCommit := r.lastCommit
-
-	var err error
-	// Attempt to pull at most numRetries times
-	for i := 0; i < numRetries; i++ {
-		if err = r.pull(); err == nil {
-			break
-		}
-		logger().Println(err)
-	}
-
-	if err != nil {
-		return err
-	}
-
-	// check if there are new changes,
-	// then execute post pull command
-	if r.lastCommit == lastCommit {
-		logger().Println("No new changes.")
-		return nil
-	}
-	return r.postPullCommand()
-}
-
-// Pull performs git clone, or git pull if repository exists
-func (r *Repo) pull() error {
-	params := []string{"clone", "-b", r.Branch, r.URL, r.Path}
-	if r.pulled {
-		params = []string{"pull", "origin", r.Branch}
-	}
-
-	// if key is specified, pull using ssh key
-	if r.KeyPath != "" {
-		return r.pullWithKey(params)
-	}
-
-	dir := ""
-	if r.pulled {
-		dir = r.Path
-	}
-
-	var err error
-	if err = runCmd(gitBinary, params, dir); err == nil {
-		r.pulled = true
-		r.lastPull = time.Now()
-		logger().Printf("%v pulled.\n", r.URL)
-		r.lastCommit, err = r.getMostRecentCommit()
-	}
-	return err
-}
-
-// pullWithKey is used for private repositories and requires an ssh key.
-// Note: currently only limited to Linux and OSX.
-func (r *Repo) pullWithKey(params []string) error {
-	var gitSSH, script gitos.File
-	// ensure temporary files deleted after usage
-	defer func() {
-		if gitSSH != nil {
-			gos.Remove(gitSSH.Name())
-		}
-		if script != nil {
-			gos.Remove(script.Name())
-		}
-	}()
-
-	var err error
-	// write git.sh script to temp file
-	gitSSH, err = writeScriptFile(gitWrapperScript())
-	if err != nil {
-		return err
-	}
-
-	// write git clone bash script to file
-	script, err = writeScriptFile(bashScript(gitSSH.Name(), r, params))
-	if err != nil {
-		return err
-	}
-
-	dir := ""
-	if r.pulled {
-		dir = r.Path
-	}
-
-	if err = runCmd(script.Name(), nil, dir); err == nil {
-		r.pulled = true
-		r.lastPull = time.Now()
-		logger().Printf("%v pulled.\n", r.URL)
-		r.lastCommit, err = r.getMostRecentCommit()
-	}
-	return err
-}
-
-// Prepare prepares for a git pull
-// and validates the configured directory
-func (r *Repo) Prepare() error {
-	// check if directory exists or is empty
-	// if not, create directory
-	fs, err := gos.ReadDir(r.Path)
-	if err != nil || len(fs) == 0 {
-		return gos.MkdirAll(r.Path, os.FileMode(0755))
-	}
-
-	// validate git repo
-	isGit := false
-	for _, f := range fs {
-		if f.IsDir() && f.Name() == ".git" {
-			isGit = true
-			break
-		}
-	}
-
-	if isGit {
-		// check if same repository
-		var repoURL string
-		if repoURL, err = r.getRepoURL(); err == nil {
-			// add .git suffix if missing for adequate comparison.
-			if !strings.HasSuffix(repoURL, ".git") {
-				repoURL += ".git"
-			}
-			if repoURL == r.URL {
-				r.pulled = true
-				return nil
-			}
-		}
-		if err != nil {
-			return fmt.Errorf("Cannot retrieve repo url for %v Error: %v", r.Path, err)
-		}
-		return fmt.Errorf("Another git repo '%v' exists at %v", repoURL, r.Path)
-	}
-	return fmt.Errorf("Cannot git clone into %v, directory not empty.", r.Path)
-}
-
-// getMostRecentCommit gets the hash of the most recent commit to the
-// repository. Useful for checking if changes occur.
-func (r *Repo) getMostRecentCommit() (string, error) {
-	command := gitBinary + ` --no-pager log -n 1 --pretty=format:"%H"`
-	c, args, err := middleware.SplitCommandAndArgs(command)
-	if err != nil {
-		return "", err
-	}
-	return runCmdOutput(c, args, r.Path)
-}
-
-// getRepoURL retrieves remote origin url for the git repository at path
-func (r *Repo) getRepoURL() (string, error) {
-	_, err := gos.Stat(r.Path)
-	if err != nil {
-		return "", err
-	}
-	args := []string{"config", "--get", "remote.origin.url"}
-	return runCmdOutput(gitBinary, args, r.Path)
-}
-
-// postPullCommand executes r.Then.
-// It is trigged after successful git pull
-func (r *Repo) postPullCommand() error {
-	if r.Then == "" {
-		return nil
-	}
-	c, args, err := middleware.SplitCommandAndArgs(r.Then)
-	if err != nil {
-		return err
-	}
-
-	if err = runCmd(c, args, r.Path); err == nil {
-		logger().Printf("Command %v successful.\n", r.Then)
-	}
-	return err
-}
-
-// Init validates git installation, locates the git executable
-// binary in PATH and check for available shell to use.
-func Init() error {
-	// prevent concurrent call
-	initMutex.Lock()
-	defer initMutex.Unlock()
-
-	// if validation has been done before and binary located in
-	// PATH, return.
-	if gitBinary != "" {
-		return nil
-	}
-
-	// locate git binary in path
-	var err error
-	if gitBinary, err = gos.LookPath("git"); err != nil {
-		return fmt.Errorf("Git middleware requires git installed. Cannot find git binary in PATH")
-	}
-
-	// locate bash in PATH. If not found, fallback to sh.
-	// If neither is found, return error.
-	shell = "bash"
-	if _, err = gos.LookPath("bash"); err != nil {
-		shell = "sh"
-		if _, err = gos.LookPath("sh"); err != nil {
-			return fmt.Errorf("Git middleware requires either bash or sh.")
-		}
-	}
-	return nil
-}
-
-// runCmd is a helper function to run commands.
-// It runs command with args from directory at dir.
-// The executed process outputs to os.Stderr
-func runCmd(command string, args []string, dir string) error {
-	cmd := gos.Command(command, args...)
-	cmd.Stdout(os.Stderr)
-	cmd.Stderr(os.Stderr)
-	cmd.Dir(dir)
-	if err := cmd.Start(); err != nil {
-		return err
-	}
-	return cmd.Wait()
-}
-
-// runCmdOutput is a helper function to run commands and return output.
-// It runs command with args from directory at dir.
-// If successful, returns output and nil error
-func runCmdOutput(command string, args []string, dir string) (string, error) {
-	cmd := gos.Command(command, args...)
-	cmd.Dir(dir)
-	var err error
-	if output, err := cmd.Output(); err == nil {
-		return string(bytes.TrimSpace(output)), nil
-	}
-	return "", err
-}
-
-// writeScriptFile writes content to a temporary file.
-// It changes the temporary file mode to executable and
-// closes it to prepare it for execution.
-func writeScriptFile(content []byte) (file gitos.File, err error) {
-	if file, err = gos.TempFile("", "caddy"); err != nil {
-		return nil, err
-	}
-	if _, err = file.Write(content); err != nil {
-		return nil, err
-	}
-	if err = file.Chmod(os.FileMode(0755)); err != nil {
-		return nil, err
-	}
-	return file, file.Close()
-}
-
-// gitWrapperScript forms content for git.sh script
-func gitWrapperScript() []byte {
-	return []byte(fmt.Sprintf(`#!/bin/%v
-
-# The MIT License (MIT)
-# Copyright (c) 2013 Alvin Abad
-
-if [ $# -eq 0 ]; then
-    echo "Git wrapper script that can specify an ssh-key file
-Usage:
-    git.sh -i ssh-key-file git-command
-    "
-    exit 1
-fi
-
-# remove temporary file on exit
-trap 'rm -f /tmp/.git_ssh.$$' 0
-
-if [ "$1" = "-i" ]; then
-    SSH_KEY=$2; shift; shift
-    echo "ssh -i $SSH_KEY \$@" > /tmp/.git_ssh.$$
-    chmod +x /tmp/.git_ssh.$$
-    export GIT_SSH=/tmp/.git_ssh.$$
-fi
-
-# in case the git command is repeated
-[ "$1" = "git" ] && shift
-
-# Run the git command
-%v "$@"
-
-`, shell, gitBinary))
-}
-
-// bashScript forms content of bash script to clone or update a repo using ssh
-func bashScript(gitShPath string, repo *Repo, params []string) []byte {
-	return []byte(fmt.Sprintf(`#!/bin/%v
-
-mkdir -p ~/.ssh;
-touch ~/.ssh/known_hosts;
-ssh-keyscan -t rsa,dsa %v 2>&1 | sort -u - ~/.ssh/known_hosts > ~/.ssh/tmp_hosts;
-cat ~/.ssh/tmp_hosts >> ~/.ssh/known_hosts;
-%v -i %v %v;
-`, shell, repo.Host, gitShPath, repo.KeyPath, strings.Join(params, " ")))
-}

middleware/git/git_test.go

@@ -1,218 +0,0 @@
-package git
-
-import (
-	"io/ioutil"
-	"log"
-	"testing"
-	"time"
-
-	"github.com/mholt/caddy/middleware/git/gittest"
-)
-
-// init sets the OS used to fakeOS.
-func init() {
-	SetOS(gittest.FakeOS)
-}
-
-func check(t *testing.T, err error) {
-	if err != nil {
-		t.Errorf("Error not expected but found %v", err)
-	}
-}
-
-func TestInit(t *testing.T) {
-	err := Init()
-	check(t, err)
-}
-
-func TestHelpers(t *testing.T) {
-	f, err := writeScriptFile([]byte("script"))
-	check(t, err)
-	var b [6]byte
-	_, err = f.Read(b[:])
-	check(t, err)
-	if string(b[:]) != "script" {
-		t.Errorf("Expected script found %v", string(b[:]))
-	}
-
-	out, err := runCmdOutput(gitBinary, []string{"-version"}, "")
-	check(t, err)
-	if out != gittest.CmdOutput {
-		t.Errorf("Expected %v found %v", gittest.CmdOutput, out)
-	}
-
-	err = runCmd(gitBinary, []string{"-version"}, "")
-	check(t, err)
-
-	wScript := gitWrapperScript()
-	if string(wScript) != expectedWrapperScript {
-		t.Errorf("Expected %v found %v", expectedWrapperScript, string(wScript))
-	}
-
-	f, err = writeScriptFile(wScript)
-	check(t, err)
-
-	repo := &Repo{Host: "github.com", KeyPath: "~/.key"}
-	script := string(bashScript(f.Name(), repo, []string{"clone", "git@github.com/repo/user"}))
-	if script != expectedBashScript {
-		t.Errorf("Expected %v found %v", expectedBashScript, script)
-	}
-}
-
-func TestGit(t *testing.T) {
-	// prepare
-	repos := []*Repo{
-		nil,
-		&Repo{Path: "gitdir", URL: "success.git"},
-	}
-	for _, r := range repos {
-		repo := createRepo(r)
-		err := repo.Prepare()
-		check(t, err)
-	}
-
-	// pull with success
-	logFile := gittest.Open("file")
-	Logger = log.New(logFile, "", 0)
-	tests := []struct {
-		repo   *Repo
-		output string
-	}{
-		{
-			&Repo{Path: "gitdir", URL: "git@github.com:user/repo.git", KeyPath: "~/.key", Then: "echo Hello"},
-			`git@github.com:user/repo.git pulled.
-Command echo Hello successful.
-`,
-		},
-		{
-			&Repo{Path: "gitdir", URL: "https://github.com/user/repo.git", Then: "echo Hello"},
-			`https://github.com/user/repo.git pulled.
-Command echo Hello successful.
-`,
-		},
-		{
-			&Repo{URL: "git@github.com:user/repo"},
-			`git@github.com:user/repo pulled.
-`,
-		},
-	}
-
-	for i, test := range tests {
-		gittest.CmdOutput = test.repo.URL
-
-		test.repo = createRepo(test.repo)
-
-		err := test.repo.Prepare()
-		check(t, err)
-
-		err = test.repo.Pull()
-		check(t, err)
-
-		out, err := ioutil.ReadAll(logFile)
-		check(t, err)
-		if test.output != string(out) {
-			t.Errorf("Pull with Success %v: Expected %v found %v", i, test.output, string(out))
-		}
-	}
-
-	// pull with error
-	repos = []*Repo{
-		&Repo{Path: "gitdir", URL: "http://github.com:u/repo.git"},
-		&Repo{Path: "gitdir", URL: "https://github.com/user/repo.git", Then: "echo Hello"},
-		&Repo{Path: "gitdir"},
-		&Repo{Path: "gitdir", KeyPath: ".key"},
-	}
-
-	gittest.CmdOutput = "git@github.com:u1/repo.git"
-	for i, repo := range repos {
-		repo = createRepo(repo)
-
-		err := repo.Prepare()
-		if err == nil {
-			t.Errorf("Pull with Error %v: Error expected but not found %v", i, err)
-			continue
-		}
-
-		expected := "Another git repo 'git@github.com:u1/repo.git' exists at gitdir"
-		if expected != err.Error() {
-			t.Errorf("Pull with Error %v: Expected %v found %v", i, expected, err.Error())
-		}
-	}
-
-}
-
-func createRepo(r *Repo) *Repo {
-	repo := &Repo{
-		URL:      "git@github.com/user/test",
-		Path:     ".",
-		Host:     "github.com",
-		Branch:   "master",
-		Interval: time.Second * 60,
-	}
-	if r == nil {
-		return repo
-	}
-	if r.Branch != "" {
-		repo.Branch = r.Branch
-	}
-	if r.Host != "" {
-		repo.Branch = r.Branch
-	}
-	if r.Interval != 0 {
-		repo.Interval = r.Interval
-	}
-	if r.KeyPath != "" {
-		repo.KeyPath = r.KeyPath
-	}
-	if r.Path != "" {
-		repo.Path = r.Path
-	}
-	if r.Then != "" {
-		repo.Then = r.Then
-	}
-	if r.URL != "" {
-		repo.URL = r.URL
-	}
-
-	return repo
-}
-
-var expectedBashScript = `#!/bin/bash
-
-mkdir -p ~/.ssh;
-touch ~/.ssh/known_hosts;
-ssh-keyscan -t rsa,dsa github.com 2>&1 | sort -u - ~/.ssh/known_hosts > ~/.ssh/tmp_hosts;
-cat ~/.ssh/tmp_hosts >> ~/.ssh/known_hosts;
-` + gittest.TempFileName + ` -i ~/.key clone git@github.com/repo/user;
-`
-
-var expectedWrapperScript = `#!/bin/bash
-
-# The MIT License (MIT)
-# Copyright (c) 2013 Alvin Abad
-
-if [ $# -eq 0 ]; then
-    echo "Git wrapper script that can specify an ssh-key file
-Usage:
-    git.sh -i ssh-key-file git-command
-    "
-    exit 1
-fi
-
-# remove temporary file on exit
-trap 'rm -f /tmp/.git_ssh.$$' 0
-
-if [ "$1" = "-i" ]; then
-    SSH_KEY=$2; shift; shift
-    echo "ssh -i $SSH_KEY \$@" > /tmp/.git_ssh.$$
-    chmod +x /tmp/.git_ssh.$$
-    export GIT_SSH=/tmp/.git_ssh.$$
-fi
-
-# in case the git command is repeated
-[ "$1" = "git" ] && shift
-
-# Run the git command
-/usr/bin/git "$@"
-
-`

middleware/git/gitos/gitos.go

@@ -1,160 +0,0 @@
-package gitos
-
-import (
-	"io"
-	"io/ioutil"
-	"os"
-	"os/exec"
-)
-
-// File is an abstraction for file (os.File).
-type File interface {
-	// Name returns the name of the file
-	Name() string
-
-	// Stat returns the FileInfo structure describing file.
-	Stat() (os.FileInfo, error)
-
-	// Close closes the File, rendering it unusable for I/O.
-	Close() error
-
-	// Chmod changes the mode of the file.
-	Chmod(os.FileMode) error
-
-	// Read reads up to len(b) bytes from the File. It returns the number of
-	// bytes read and an error, if any.
-	Read([]byte) (int, error)
-
-	// Write writes len(b) bytes to the File. It returns the number of bytes
-	// written and an error, if any.
-	Write([]byte) (int, error)
-}
-
-// Cmd is an abstraction for external commands (os.Cmd).
-type Cmd interface {
-	// Run starts the specified command and waits for it to complete.
-	Run() error
-
-	// Start starts the specified command but does not wait for it to complete.
-	Start() error
-
-	// Wait waits for the command to exit. It must have been started by Start.
-	Wait() error
-
-	// Output runs the command and returns its standard output.
-	Output() ([]byte, error)
-
-	// Dir sets the working directory of the command.
-	Dir(string)
-
-	// Stdin sets the process's standard input.
-	Stdin(io.Reader)
-
-	// Stdout sets the process's standard output.
-	Stdout(io.Writer)
-
-	// Stderr sets the process's standard output.
-	Stderr(io.Writer)
-}
-
-// gitCmd represents external commands executed by git.
-type gitCmd struct {
-	*exec.Cmd
-}
-
-// Dir sets the working directory of the command.
-func (g *gitCmd) Dir(dir string) {
-	g.Cmd.Dir = dir
-}
-
-// Stdin sets the process's standard input.
-func (g *gitCmd) Stdin(stdin io.Reader) {
-	g.Cmd.Stdin = stdin
-}
-
-// Stdout sets the process's standard output.
-func (g *gitCmd) Stdout(stdout io.Writer) {
-	g.Cmd.Stdout = stdout
-}
-
-// Stderr sets the process's standard output.
-func (g *gitCmd) Stderr(stderr io.Writer) {
-	g.Cmd.Stderr = stderr
-}
-
-// OS is an abstraction for required OS level functions.
-type OS interface {
-	// Command returns the Cmd to execute the named program with the
-	// given arguments.
-	Command(string, ...string) Cmd
-
-	// Mkdir creates a new directory with the specified name and permission
-	// bits.
-	Mkdir(string, os.FileMode) error
-
-	// MkdirAll creates a directory named path, along with any necessary
-	// parents.
-	MkdirAll(string, os.FileMode) error
-
-	// Stat returns a FileInfo describing the named file.
-	Stat(string) (os.FileInfo, error)
-
-	// Remove removes the named file or directory.
-	Remove(string) error
-
-	// ReadDir reads the directory named by dirname and returns a list of
-	// directory entries.
-	ReadDir(string) ([]os.FileInfo, error)
-
-	// LookPath searches for an executable binary named file in the directories
-	// named by the PATH environment variable.
-	LookPath(string) (string, error)
-
-	// TempFile creates a new temporary file in the directory dir with a name
-	// beginning with prefix, opens the file for reading and writing, and
-	// returns the resulting File.
-	TempFile(string, string) (File, error)
-}
-
-// GitOS is the implementation of OS for git.
-type GitOS struct{}
-
-// Mkdir calls os.Mkdir.
-func (g GitOS) Mkdir(name string, perm os.FileMode) error {
-	return os.Mkdir(name, perm)
-}
-
-// MkdirAll calls os.MkdirAll.
-func (g GitOS) MkdirAll(path string, perm os.FileMode) error {
-	return os.MkdirAll(path, perm)
-}
-
-// Stat calls os.Stat.
-func (g GitOS) Stat(name string) (os.FileInfo, error) {
-	return os.Stat(name)
-}
-
-// Remove calls os.Remove.
-func (g GitOS) Remove(name string) error {
-	return os.Remove(name)
-}
-
-// LookPath calls exec.LookPath.
-func (g GitOS) LookPath(file string) (string, error) {
-	return exec.LookPath(file)
-}
-
-// TempFile calls ioutil.TempFile.
-func (g GitOS) TempFile(dir, prefix string) (File, error) {
-	return ioutil.TempFile(dir, prefix)
-}
-
-// ReadDir calls ioutil.ReadDir.
-func (g GitOS) ReadDir(dirname string) ([]os.FileInfo, error) {
-	return ioutil.ReadDir(dirname)
-}
-
-// Command calls exec.Command.
-func (g GitOS) Command(name string, args ...string) Cmd {
-	return &gitCmd{exec.Command(name, args...)}
-}

middleware/git/gittest/gittest.go

@@ -1,167 +0,0 @@
-// Package gittest is a test package for the git middleware.
-// It implements a mock gitos.OS, gitos.Cmd and gitos.File.
-package gittest
-
-import (
-	"io"
-	"os"
-	"time"
-
-	"github.com/mholt/caddy/middleware/git/gitos"
-)
-
-// FakeOS implements a mock gitos.OS, gitos.Cmd and gitos.File.
-var FakeOS = fakeOS{}
-
-// CmdOutput is the output of any call to the mocked gitos.Cmd's Output().
-var CmdOutput = "success"
-
-// TempFileName is the name of any file returned by mocked gitos.OS's TempFile().
-var TempFileName = "tempfile"
-
-// dirs mocks a fake git dir if filename is "gitdir".
-var dirs = map[string][]os.FileInfo{
-	"gitdir": {
-		fakeInfo{name: ".git", dir: true},
-	},
-}
-
-// Open creates a new mock gitos.File.
-func Open(name string) gitos.File {
-	return &fakeFile{name: name}
-}
-
-// fakeFile is a mock gitos.File.
-type fakeFile struct {
-	name    string
-	dir     bool
-	content []byte
-	info    fakeInfo
-}
-
-func (f fakeFile) Name() string {
-	return f.name
-}
-
-func (f fakeFile) Stat() (os.FileInfo, error) {
-	return fakeInfo{name: f.name}, nil
-}
-
-func (f fakeFile) Close() error {
-	return nil
-}
-
-func (f fakeFile) Chmod(mode os.FileMode) error {
-	f.info.mode = mode
-	return nil
-}
-
-func (f *fakeFile) Read(b []byte) (int, error) {
-	if len(f.content) == 0 {
-		return 0, io.EOF
-	}
-	n := copy(b, f.content)
-	f.content = f.content[n:]
-	return n, nil
-}
-
-func (f *fakeFile) Write(b []byte) (int, error) {
-	f.content = append(f.content, b...)
-	return len(b), nil
-}
-
-// fakeCmd is a mock git.Cmd.
-type fakeCmd struct{}
-
-func (f fakeCmd) Run() error {
-	return nil
-}
-
-func (f fakeCmd) Start() error {
-	return nil
-}
-
-func (f fakeCmd) Wait() error {
-	return nil
-}
-
-func (f fakeCmd) Output() ([]byte, error) {
-	return []byte(CmdOutput), nil
-}
-
-func (f fakeCmd) Dir(dir string) {}
-
-func (f fakeCmd) Stdin(stdin io.Reader) {}
-
-func (f fakeCmd) Stdout(stdout io.Writer) {}
-
-func (f fakeCmd) Stderr(stderr io.Writer) {}
-
-// fakeInfo is a mock os.FileInfo.
-type fakeInfo struct {
-	name string
-	dir  bool
-	mode os.FileMode
-}
-
-func (f fakeInfo) Name() string {
-	return f.name
-}
-
-func (f fakeInfo) Size() int64 {
-	return 1024
-}
-
-func (f fakeInfo) Mode() os.FileMode {
-	return f.mode
-}
-
-func (f fakeInfo) ModTime() time.Time {
-	return time.Now().Truncate(time.Hour)
-}
-
-func (f fakeInfo) IsDir() bool {
-	return f.dir
-}
-
-func (f fakeInfo) Sys() interface{} {
-	return nil
-}
-
-// fakeOS is a mock git.OS.
-type fakeOS struct{}
-
-func (f fakeOS) Mkdir(name string, perm os.FileMode) error {
-	return nil
-}
-
-func (f fakeOS) MkdirAll(path string, perm os.FileMode) error {
-	return nil
-}
-
-func (f fakeOS) Stat(name string) (os.FileInfo, error) {
-	return fakeInfo{name: name}, nil
-}
-
-func (f fakeOS) Remove(name string) error {
-	return nil
-}
-
-func (f fakeOS) LookPath(file string) (string, error) {
-	return "/usr/bin/" + file, nil
-}
-
-func (f fakeOS) TempFile(dir, prefix string) (gitos.File, error) {
-	return &fakeFile{name: TempFileName, info: fakeInfo{name: TempFileName}}, nil
-}
-
-func (f fakeOS) ReadDir(dirname string) ([]os.FileInfo, error) {
-	if f, ok := dirs[dirname]; ok {
-		return f, nil
-	}
-	return nil, nil
-}
-
-func (f fakeOS) Command(name string, args ...string) gitos.Cmd {
-	return fakeCmd{}
-}

middleware/git/os.go

@@ -1,12 +0,0 @@
-package git
-
-import "github.com/mholt/caddy/middleware/git/gitos"
-
-// gos is the OS used by git.
-var gos gitos.OS = gitos.GitOS{}
-
-// SetOS sets the OS to be used. Intended to be used for tests
-// to abstract OS level git actions.
-func SetOS(os gitos.OS) {
-	gos = os
-}

middleware/gzip/gzip.go

@@ -1,75 +0,0 @@
-// Package gzip provides a simple middleware layer that performs
-// gzip compression on the response.
-package gzip
-
-import (
-	"compress/gzip"
-	"fmt"
-	"io"
-	"net/http"
-	"strings"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-// Gzip is a middleware type which gzips HTTP responses. It is
-// imperative that any handler which writes to a gzipped response
-// specifies the Content-Type, otherwise some clients will assume
-// application/x-gzip and try to download a file.
-type Gzip struct {
-	Next middleware.Handler
-}
-
-// ServeHTTP serves a gzipped response if the client supports it.
-func (g Gzip) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-	if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
-		return g.Next.ServeHTTP(w, r)
-	}
-
-	// Delete this header so gzipping isn't repeated later in the chain
-	r.Header.Del("Accept-Encoding")
-
-	w.Header().Set("Content-Encoding", "gzip")
-	gzipWriter := gzip.NewWriter(w)
-	defer gzipWriter.Close()
-	gz := gzipResponseWriter{Writer: gzipWriter, ResponseWriter: w}
-
-	// Any response in forward middleware will now be compressed
-	status, err := g.Next.ServeHTTP(gz, r)
-
-	// If there was an error that remained unhandled, we need
-	// to send something back before gzipWriter gets closed at
-	// the return of this method!
-	if status >= 400 {
-		gz.Header().Set("Content-Type", "text/plain") // very necessary
-		gz.WriteHeader(status)
-		fmt.Fprintf(gz, "%d %s", status, http.StatusText(status))
-		return 0, err
-	}
-	return status, err
-}
-
-// gzipResponeWriter wraps the underlying Write method
-// with a gzip.Writer to compress the output.
-type gzipResponseWriter struct {
-	io.Writer
-	http.ResponseWriter
-}
-
-// WriteHeader wraps the underlying WriteHeader method to prevent
-// problems with conflicting headers from proxied backends. For
-// example, a backend system that calculates Content-Length would
-// be wrong because it doesn't know it's being gzipped.
-func (w gzipResponseWriter) WriteHeader(code int) {
-	w.Header().Del("Content-Length")
-	w.ResponseWriter.WriteHeader(code)
-}
-
-// Write wraps the underlying Write method to do compression.
-func (w gzipResponseWriter) Write(b []byte) (int, error) {
-	if w.Header().Get("Content-Type") == "" {
-		w.Header().Set("Content-Type", http.DetectContentType(b))
-	}
-	n, err := w.Writer.Write(b)
-	return n, err
-}

middleware/headers/headers.go

@@ -1,50 +0,0 @@
-// Package headers provides middleware that appends headers to
-// requests based on a set of configuration rules that define
-// which routes receive which headers.
-package headers
-
-import (
-	"net/http"
-	"strings"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-// Headers is middleware that adds headers to the responses
-// for requests matching a certain path.
-type Headers struct {
-	Next  middleware.Handler
-	Rules []Rule
-}
-
-// ServeHTTP implements the middleware.Handler interface and serves requests,
-// setting headers on the response according to the configured rules.
-func (h Headers) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-	for _, rule := range h.Rules {
-		if middleware.Path(r.URL.Path).Matches(rule.Path) {
-			for _, header := range rule.Headers {
-				if strings.HasPrefix(header.Name, "-") {
-					w.Header().Del(strings.TrimLeft(header.Name, "-"))
-				} else {
-					w.Header().Set(header.Name, header.Value)
-				}
-			}
-		}
-	}
-	return h.Next.ServeHTTP(w, r)
-}
-
-type (
-	// Rule groups a slice of HTTP headers by a URL pattern.
-	// TODO: use http.Header type instead?
-	Rule struct {
-		Path    string
-		Headers []Header
-	}
-
-	// Header represents a single HTTP header, simply a name and value.
-	Header struct {
-		Name  string
-		Value string
-	}
-)

middleware/headers/headers_test.go

@@ -1,50 +0,0 @@
-package headers
-
-import (
-	"net/http"
-	"net/http/httptest"
-	"testing"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-func TestHeaders(t *testing.T) {
-	for i, test := range []struct {
-		from  string
-		name  string
-		value string
-	}{
-		{"/a", "Foo", "Bar"},
-		{"/a", "Bar", ""},
-		{"/a", "Baz", ""},
-		{"/b", "Foo", ""},
-		{"/b", "Bar", "Removed in /a"},
-	} {
-		he := Headers{
-			Next: middleware.HandlerFunc(func(w http.ResponseWriter, r *http.Request) (int, error) {
-				return 0, nil
-			}),
-			Rules: []Rule{
-				{Path: "/a", Headers: []Header{
-					{Name: "Foo", Value: "Bar"},
-					{Name: "-Bar"},
-				}},
-			},
-		}
-
-		req, err := http.NewRequest("GET", test.from, nil)
-		if err != nil {
-			t.Fatalf("Test %d: Could not create HTTP request: %v", i, err)
-		}
-
-		rec := httptest.NewRecorder()
-		rec.Header().Set("Bar", "Removed in /a")
-
-		he.ServeHTTP(rec, req)
-
-		if got := rec.Header().Get(test.name); got != test.value {
-			t.Errorf("Test %d: Expected %s header to be %q but was %q",
-				i, test.name, test.value, got)
-		}
-	}
-}

middleware/internal/internal.go

@@ -1,90 +0,0 @@
-// Package internal provides a simple middleware that (a) prevents access
-// to internal locations and (b) allows to return files from internal location
-// by setting a special header, e.g. in a proxy response.
-package internal
-
-import (
-	"net/http"
-
-	"github.com/mholt/caddy/middleware"
-)
-
-// Internal middleware protects internal locations from external requests -
-// but allows access from the inside by using a special HTTP header.
-type Internal struct {
-	Next  middleware.Handler
-	Paths []string
-}
-
-const (
-	redirectHeader   string = "X-Accel-Redirect"
-	maxRedirectCount int    = 10
-)
-
-func isInternalRedirect(w http.ResponseWriter) bool {
-	return w.Header().Get(redirectHeader) != ""
-}
-
-// ServeHTTP implements the middlware.Handler interface.
-func (i Internal) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
-
-	// Internal location requested? -> Not found.
-	for _, prefix := range i.Paths {
-		if middleware.Path(r.URL.Path).Matches(prefix) {
-			return http.StatusNotFound, nil
-		}
-	}
-
-	// Use internal response writer to ignore responses that will be
-	// redirected to internal locations
-	iw := internalResponseWriter{ResponseWriter: w}
-	status, err := i.Next.ServeHTTP(iw, r)
-
-	for c := 0; c < maxRedirectCount && isInternalRedirect(iw); c++ {
-		// Redirect - adapt request URL path and send it again
-		// "down the chain"
-		r.URL.Path = iw.Header().Get(redirectHeader)
-		iw.ClearHeader()
-
-		status, err = i.Next.ServeHTTP(iw, r)
-	}
-
-	if isInternalRedirect(iw) {
-		// Too many redirect cycles
-		iw.ClearHeader()
-		return http.StatusInternalServerError, nil
-	}
-
-	return status, err
-}
-
-// internalResponseWriter wraps the underlying http.ResponseWriter and ignores
-// calls to Write and WriteHeader if the response should be redirected to an
-// internal location.
-type internalResponseWriter struct {
-	http.ResponseWriter
-}
-
-// ClearHeader removes all header fields that are already set.
-func (w internalResponseWriter) ClearHeader() {
-	for k := range w.Header() {
-		w.Header().Del(k)
-	}
-}
-
-// WriteHeader ignores the call if the response should be redirected to an
-// internal location.
-func (w internalResponseWriter) WriteHeader(code int) {
-	if !isInternalRedirect(w) {
-		w.ResponseWriter.WriteHeader(code)
-	}
-}
-
-// Write ignores the call if the response should be redirected to an internal
-// location.
-func (w internalResponseWriter) Write(b []byte) (int, error) {
-	if isInternalRedirect(w) {
-		return 0, nil
-	}
-	return w.ResponseWriter.Write(b)
-}