Add test for 5b727bde2992e7cb9987208453db42ae6e1c6e1e

* fix(provisioning): remove app from apps map when its provision failed

* Clean up failed app provisioning with defer

* fix(provisioning): record apps that failed to provision with their error

* save the error when an app fails to initialize and return this error when this app is requested by a module

---------

Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: WeidiDeng <weidi_deng@icloud.com>

.editorconfig

@@ -1,5 +1,5 @@
 [*]
 end_of_line = lf
 
-[caddytest/integration/caddyfile_adapt/*.txt]
+[caddytest/integration/caddyfile_adapt/*.caddyfiletest]
 indent_style = tab

.github/CONTRIBUTING.md

@@ -1,7 +1,7 @@
 Contributing to Caddy
 =====================
 
-Welcome! Thank you for choosing to be a part of our community. Caddy wouldn't be great without your involvement!
+Welcome! Thank you for choosing to be a part of our community. Caddy wouldn't be nearly as excellent without your involvement!
 
 For starters, we invite you to join [the Caddy forum](https://caddy.community) where you can hang out with other Caddy users and developers.
 
@@ -25,7 +25,7 @@ Other menu items:
 
 You can have a huge impact on the project by helping with its code. To contribute code to Caddy, first submit or comment in an issue to discuss your contribution, then open a [pull request](https://github.com/caddyserver/caddy/pulls) (PR). If you're new to our community, that's okay: **we gladly welcome pull requests from anyone, regardless of your native language or coding experience.** You can get familiar with Caddy's code base by using [code search at Sourcegraph](https://sourcegraph.com/github.com/caddyserver/caddy).
 
-We hold contributions to a high standard for quality :bowtie:, so don't be surprised if we ask for revisions—even if it seems small or insignificant. Please don't take it personally. :blue_heart: If your change is on the right track, we can guide you to make it mergable.
+We hold contributions to a high standard for quality :bowtie:, so don't be surprised if we ask for revisions—even if it seems small or insignificant. Please don't take it personally. :blue_heart: If your change is on the right track, we can guide you to make it mergeable.
 
 Here are some of the expectations we have of contributors:
 
@@ -35,19 +35,29 @@ Here are some of the expectations we have of contributors:
 
 - **Keep related commits together in a PR.** We do want pull requests to be small, but you should also keep multiple related commits in the same PR if they rely on each other.
 
-- **Write tests.** Tests are essential! Written properly, they ensure your change works, and that other changes in the future won't break your change. CI checks should pass.
+- **Write tests.** Good, automated tests are very valuable! Written properly, they ensure your change works, and that other changes in the future won't break your change. CI checks should pass.
 
-- **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven with benchmarks or profiling.
+- **Benchmarks should be included for optimizations.** Optimizations sometimes make code harder to read or have changes that are less than obvious. They should be proven with benchmarks and profiling.
 
 - **[Squash](http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html) insignificant commits.** Every commit should be significant. Commits which merely rewrite a comment or fix a typo can be combined into another commit that has more substance. Interactive rebase can do this, or a simpler way is `git reset --soft <diverging-commit>` then `git commit -s`.
 
-- **Own your contributions.** Caddy is a growing project, and it's much better when individual contributors help maintain their change after it is merged.
+- **Be responsible for and maintain your contributions.** Caddy is a growing project, and it's much better when individual contributors help maintain their change after it is merged.
 
 - **Use comments properly.** We expect good godoc comments for package-level functions, types, and values. Comments are also useful whenever the purpose for a line of code is not obvious.
 
-- **Pull requests may still get closed.** The longer a PR stays open and idle, the more likely it is to be closed. If we haven't reviewed it in a while, it probably means the change is not a priority. Please don't take this personally, we're trying to balance a lot of tasks! If nobody else has commented or reacted to the PR, it likely means your change is useful only to you. The reality is this happens quite a bit. We don't tend to accept PRs that aren't generally helpful. For these reasons or others, the PR may get closed even after a review. We are not obligated to accept all proposed changes, even if the best justification we can give is something vague like, "It doesn't sit right." Sometimes PRs are just the wrong thing or the wrong time. Because it is open source, you can always build your own modified version of Caddy with a change you need, even if we reject it in the official repo.
+- **Pull requests may still get closed.** The longer a PR stays open and idle, the more likely it is to be closed. If we haven't reviewed it in a while, it probably means the change is not a priority. Please don't take this personally, we're trying to balance a lot of tasks! If nobody else has commented or reacted to the PR, it likely means your change is useful only to you. The reality is this happens quite a lot. We don't tend to accept PRs that aren't generally helpful. For these reasons or others, the PR may get closed even after a review. We are not obligated to accept all proposed changes, even if the best justification we can give is something vague like, "It doesn't sit right." Sometimes PRs are just the wrong thing or the wrong time. Because it is open source, you can always build your own modified version of Caddy with a change you need, even if we reject it in the official repo. Plus, because Caddy is extensible, it's possible your feature could make a great plugin instead!
 
-We often grant [collaborator status](#collaborator-instructions) to contributors who author one or more significant, high-quality PRs that are merged into the code base!
+- **You certify that you wrote and comprehend the code you submit.** The Caddy project welcomes original contributions that comply with [our CLA](https://cla-assistant.io/caddyserver/caddy), meaning that authors must be able to certify that they created or have rights to the code they are contributing. In addition, we require that code is not simply copy-pasted from Q/A sites or AI language models without full comprehension and rigorous testing. In other words: contributors are allowed to refer to communities for assistance and use AI tools such as language models for inspiration, but code which originates from or is assisted by these resources MUST be:
+
+	- Licensed for you to freely share
+	- Fully comprehended by you (be able to explain every line of code)
+	- Verified by automated tests when feasible, or thorough manual tests otherwise
+
+	We have found that current language models (LLMs, like ChatGPT) may understand code syntax and even problem spaces to an extent, but often fail in subtle ways to convey true knowledge and produce correct algorithms. Integrated tools such as GitHub Copilot and Sourcegraph Cody may be used for inspiration, but code generated by these tools still needs to meet our criteria for licensing, human comprehension, and testing. These tools may be used to help write code comments and tests as long as you can certify they are accurate and correct. Note that it is often more trouble than it's worth to certify that Copilot (for example) is not giving you code that is possibly plagiarised, unlicensed, or licensed with incompatible terms -- as the Caddy project cannot accept such contributions. If that's too difficult for you (or impossible), then we recommend using these resources only for inspiration and write your own code. Ultimately, you (the contributor) are responsible for the code you're submitting.
+
+	As a courtesy to reviewers, we kindly ask that you disclose when contributing code that was generated by an AI tool or copied from another website so we can be aware of what to look for in code review.
+
+We often grant [collaborator status](#collaborator-instructions) to contributors who author one or more significant, high-quality PRs that are merged into the code base.
 
 
 #### HOW TO MAKE A PULL REQUEST TO CADDY

.github/SECURITY.md

@@ -5,11 +5,11 @@ The Caddy project would like to make sure that it stays on top of all practicall
 
 ## Supported Versions
 
-| Version | Supported          |
-| ------- | ------------------ |
-| 2.x     | :white_check_mark: |
-| 1.x     | :x:                |
-| < 1.x   | :x:                |
+| Version  | Supported |
+| -------- | ----------|
+| 2.latest | ✔️        |
+| 1.x      | :x:       |
+| < 1.x    | :x:       |
 
 
 ## Acceptable Scope
@@ -24,7 +24,7 @@ We do not accept reports if the steps imply or require a compromised system or t
 
 Client-side exploits are out of scope. In other words, it is not a bug in Caddy if the web browser does something unsafe, even if the downloaded content was served by Caddy. (Those kinds of exploits can generally be mitigated by proper configuration of HTTP headers.) As a general rule, the content served by Caddy is not considered in scope because content is configurable by the site owner or the associated web application.
 
-Security bugs in code dependencies are out of scope. Instead, if a dependency has patched a relevant security bug, please feel free to open a public issue or pull request to update that dependency in our code.
+Security bugs in code dependencies (including Go's standard library) are out of scope. Instead, if a dependency has patched a relevant security bug, please feel free to open a public issue or pull request to update that dependency in our code.
 
 
 ## Reporting a Vulnerability
@@ -42,15 +42,15 @@ We'll need enough information to verify the bug and make a patch. To speed thing
 - Specific minimal steps to reproduce the issue from scratch
 - A working patch
 
-Please DO NOT use containers, VMs, cloud instances or services, or any other complex infrastructure in your steps. Always prefer `curl` instead of web browsers.
+Please DO NOT use containers, VMs, cloud instances or services, or any other complex infrastructure in your steps. Always prefer `curl -v` instead of web browsers.
 
 We consider publicly-registered domain names to be public information. This necessary in order to maintain the integrity of certificate transparency, public DNS, and other public trust systems. Do not redact domain names from your reports. The actual content of your domain name affects Caddy's behavior, so we need the exact domain name(s) to reproduce with, or your report will be ignored.
 
 It will speed things up if you suggest a working patch, such as a code diff, and explain why and how it works. Reports that are not actionable, do not contain enough information, are too pushy/demanding, or are not able to convince us that it is a viable and practical attack on the web server itself may be deferred to a later time or possibly ignored, depending on available resources. Priority will be given to credible, responsible reports that are constructive, specific, and actionable. (We get a lot of invalid reports.) Thank you for understanding.
 
-When you are ready, please email Matt Holt (the author) directly: matt at dyanim dot com.
+When you are ready, please submit a [new private vulnerability report](https://github.com/caddyserver/caddy/security/advisories/new).
 
-Please don't encrypt the email body. It only makes the process more complicated.
+Please don't encrypt the message. It only makes the process more complicated.
 
 Please also understand that due to our nature as an open source project, we do not have a budget to award security bounties. We can only thank you.
 

.github/dependabot.yml

@@ -0,0 +1,22 @@
+---
+version: 2
+updates:
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    open-pull-requests-limit: 1
+    groups:
+      actions-deps:
+        patterns:
+          - "*"
+    schedule:
+      interval: "monthly"
+    
+  - package-ecosystem: "gomod"
+    directory: "/"
+    open-pull-requests-limit: 1
+    groups:
+      all-updates:
+        patterns:
+          - "*"
+    schedule:
+      interval: "monthly"

.github/workflows/ci.yml

@@ -12,51 +12,71 @@ on:
       - master
       - 2.*
 
+env:
+  # https://github.com/actions/setup-go/issues/491
+  GOTOOLCHAIN: local
+
+permissions:
+  contents: read
+
 jobs:
   test:
     strategy:
       # Default is true, cancels jobs for other platforms in the matrix if one fails
       fail-fast: false
       matrix:
-        os: [ ubuntu-latest, macos-latest, windows-latest ]
-        go: [ '1.18', '1.19' ]
+        os:
+          - linux
+          - mac
+          - windows
+        go:
+          - '1.24'
 
         include:
         # Set the minimum Go patch version for the given Go minor
         # Usable via ${{ matrix.GO_SEMVER }}
-        - go: '1.18'
-          GO_SEMVER: '~1.18.4'
-
-        - go: '1.19'
-          GO_SEMVER: '~1.19.0'
+        - go: '1.24'
+          GO_SEMVER: '~1.24.1'
 
         # Set some variables per OS, usable via ${{ matrix.VAR }}
+        # OS_LABEL: the VM label from GitHub Actions (see https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories)
         # CADDY_BIN_PATH: the path to the compiled Caddy binary, for artifact publishing
         # SUCCESS: the typical value for $? per OS (Windows/pwsh returns 'True')
-        - os: ubuntu-latest
+        - os: linux
+          OS_LABEL: ubuntu-latest
           CADDY_BIN_PATH: ./cmd/caddy/caddy
           SUCCESS: 0
 
-        - os: macos-latest
+        - os: mac
+          OS_LABEL: macos-14
           CADDY_BIN_PATH: ./cmd/caddy/caddy
           SUCCESS: 0
 
-        - os: windows-latest
+        - os: windows
+          OS_LABEL: windows-latest
           CADDY_BIN_PATH: ./cmd/caddy/caddy.exe
           SUCCESS: 'True'
 
-    runs-on: ${{ matrix.os }}
-
+    runs-on: ${{ matrix.OS_LABEL }}
+    permissions:
+      contents: read
+      pull-requests: read
+      actions: write # to allow uploading artifacts and cache
     steps:
+    - name: Harden the runner (Audit all outbound calls)
+      uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
+      with:
+        egress-policy: audit
+
+    - name: Checkout code
+      uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+
     - name: Install Go
-      uses: actions/setup-go@v3
+      uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
       with:
         go-version: ${{ matrix.GO_SEMVER }}
         check-latest: true
 
-    - name: Checkout code
-      uses: actions/checkout@v3
-
     # These tools would be useful if we later decide to reinvestigate
     # publishing test/coverage reports to some tool for easier consumption
     # - name: Install test and coverage analysis tools
@@ -64,10 +84,11 @@ jobs:
     #     go get github.com/axw/gocov/gocov
     #     go get github.com/AlekSi/gocov-xml
     #     go get -u github.com/jstemmer/go-junit-report
-    #     echo "::add-path::$(go env GOPATH)/bin"
+    #     echo "$(go env GOPATH)/bin" >> $GITHUB_PATH
 
     - name: Print Go version and environment
       id: vars
+      shell: bash
       run: |
         printf "Using go at: $(which go)\n"
         printf "Go version: $(go version)\n"
@@ -77,24 +98,7 @@ jobs:
         env
         printf "Git version: $(git version)\n\n"
         # Calculate the short SHA1 hash of the git commit
-        echo "::set-output name=short_sha::$(git rev-parse --short HEAD)"
-
-    - name: Cache the build cache
-      uses: actions/cache@v2
-      with:
-        # In order:
-        # * Module download cache
-        # * Build cache (Linux)
-        # * Build cache (Mac)
-        # * Build cache (Windows)
-        path: |
-          ~/go/pkg/mod
-          ~/.cache/go-build
-          ~/Library/Caches/go-build
-          ~\AppData\Local\go-build
-        key: ${{ runner.os }}-${{ matrix.go }}-go-ci-${{ hashFiles('**/go.sum') }}
-        restore-keys: |
-          ${{ runner.os }}-${{ matrix.go }}-go-ci
+        echo "short_sha=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
 
     - name: Get dependencies
       run: |
@@ -106,13 +110,20 @@ jobs:
       env:
         CGO_ENABLED: 0
       run: |
-        go build -trimpath -ldflags="-w -s" -v
+        go build -tags nobadger,nomysql,nopgx -trimpath -ldflags="-w -s" -v
+
+    - name: Smoke test Caddy
+      working-directory: ./cmd/caddy
+      run: |
+        ./caddy start
+        ./caddy stop
 
     - name: Publish Build Artifact
-      uses: actions/upload-artifact@v1
+      uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
       with:
         name: caddy_${{ runner.os }}_go${{ matrix.go }}_${{ steps.vars.outputs.short_sha }}
         path: ${{ matrix.CADDY_BIN_PATH }}
+        compression-level: 0
 
     # Commented bits below were useful to allow the job to continue
     # even if the tests fail, so we can publish the report separately
@@ -122,8 +133,8 @@ jobs:
       # continue-on-error: true
       run: |
         # (go test -v -coverprofile=cover-profile.out -race ./... 2>&1) > test-results/test-result.out
-        go test -v -coverprofile="cover-profile.out" -short -race ./...
-        # echo "::set-output name=status::$?"
+        go test -tags nobadger,nomysql,nopgx -v -coverprofile="cover-profile.out" -short -race ./...
+        # echo "status=$?" >> $GITHUB_OUTPUT
 
     # Relevant step if we reinvestigate publishing test/coverage reports
     # - name: Prepare coverage reports
@@ -135,48 +146,103 @@ jobs:
 
     # To return the correct result even though we set 'continue-on-error: true'
     # - name: Coerce correct build result
-    #   if: matrix.os != 'windows-latest' && steps.step_test.outputs.status != ${{ matrix.SUCCESS }}
+    #   if: matrix.os != 'windows' && steps.step_test.outputs.status != ${{ matrix.SUCCESS }}
     #   run: |
     #     echo "step_test ${{ steps.step_test.outputs.status }}\n"
     #     exit 1
 
   s390x-test:
     name: test (s390x on IBM Z)
+    permissions:
+      contents: read
+      pull-requests: read
     runs-on: ubuntu-latest
-    if: github.event.pull_request.head.repo.full_name == github.repository
+    if: github.event.pull_request.head.repo.full_name == 'caddyserver/caddy' && github.actor != 'dependabot[bot]'
     continue-on-error: true  # August 2020: s390x VM is down due to weather and power issues
     steps:
-      - name: Checkout code into the Go module directory
-        uses: actions/checkout@v3
+      - name: Harden the runner (Audit all outbound calls)
+        uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
+        with:
+          egress-policy: audit
+          allowed-endpoints: ci-s390x.caddyserver.com:22
+
+      - name: Checkout code
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
       - name: Run Tests
         run: |
+          set +e
           mkdir -p ~/.ssh && echo -e "${SSH_KEY//_/\\n}" > ~/.ssh/id_ecdsa && chmod og-rwx ~/.ssh/id_ecdsa
 
           # short sha is enough?
           short_sha=$(git rev-parse --short HEAD)
 
+          # To shorten the following lines
+          ssh_opts="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"
+          ssh_host="$CI_USER@ci-s390x.caddyserver.com"
+
           # The environment is fresh, so there's no point in keeping accepting and adding the key.
-          rsync -arz -e "ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" --progress --delete --exclude '.git' . caddy-ci@ci-s390x.caddyserver.com:/var/tmp/"$short_sha"
-          ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -t caddy-ci@ci-s390x.caddyserver.com "cd /var/tmp/$short_sha; go version; go env; printf "\n\n";CGO_ENABLED=0 go test -v ./..."
+          rsync -arz -e "ssh $ssh_opts" --progress --delete --exclude '.git' . "$ssh_host":/var/tmp/"$short_sha"
+          ssh $ssh_opts -t "$ssh_host" bash <<EOF
+          cd /var/tmp/$short_sha
+          go version
+          go env
+          printf "\n\n"
+          retries=3
+          exit_code=0
+          while ((retries > 0)); do
+            CGO_ENABLED=0 go test -p 1 -tags nobadger,nomysql,nopgx -v ./...
+            exit_code=$?
+            if ((exit_code == 0)); then
+              break
+            fi
+            echo "\n\nTest failed: \$exit_code, retrying..."
+            ((retries--))
+          done
+          echo "Remote exit code: \$exit_code"
+          exit \$exit_code
+          EOF
           test_result=$?
 
           # There's no need leaving the files around
-          ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null caddy-ci@ci-s390x.caddyserver.com "rm -rf /var/tmp/'$short_sha'"
+          ssh $ssh_opts "$ssh_host" "rm -rf /var/tmp/'$short_sha'"
 
           echo "Test exit code: $test_result"
           exit $test_result
         env:
           SSH_KEY: ${{ secrets.S390X_SSH_KEY }}
+          CI_USER: ${{ secrets.CI_USER }}
 
   goreleaser-check:
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: read
+    if: github.event.pull_request.head.repo.full_name == 'caddyserver/caddy' && github.actor != 'dependabot[bot]'
     steps:
-      - name: checkout
-        uses: actions/checkout@v3
+      - name: Harden the runner (Audit all outbound calls)
+        uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
+        with:
+          egress-policy: audit
+
+      - name: Checkout code
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
       
-      - uses: goreleaser/goreleaser-action@v2
+      - uses: goreleaser/goreleaser-action@9c156ee8a17a598857849441385a2041ef570552 # v6.3.0
         with:
           version: latest
           args: check
+      - name: Install Go
+        uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+        with:
+          go-version: "~1.24"
+          check-latest: true
+      - name: Install xcaddy
+        run: |
+          go install github.com/caddyserver/xcaddy/cmd/xcaddy@latest
+          xcaddy version
+      - uses: goreleaser/goreleaser-action@9c156ee8a17a598857849441385a2041ef570552 # v6.3.0
+        with:
+          version: latest
+          args: build --single-target --snapshot
         env:
-          TAG: ${{ steps.vars.outputs.version_tag }}
+          TAG: ${{ github.head_ref || github.ref_name }}

.github/workflows/cross-build.yml

@@ -10,25 +10,54 @@ on:
       - master
       - 2.*
 
+env:
+  # https://github.com/actions/setup-go/issues/491
+  GOTOOLCHAIN: local
+
+permissions:
+  contents: read
+
 jobs:
-  cross-build-test:
+  build:
     strategy:
       fail-fast: false
       matrix:
-        goos: ['android', 'linux', 'solaris', 'illumos', 'dragonfly', 'freebsd', 'openbsd', 'plan9', 'windows', 'darwin', 'netbsd']
-        go: [ '1.19' ]
+        goos:
+          - 'aix'
+          - 'linux'
+          - 'solaris'
+          - 'illumos'
+          - 'dragonfly'
+          - 'freebsd'
+          - 'openbsd'
+          - 'windows'
+          - 'darwin'
+          - 'netbsd'
+        go:
+          - '1.24'
 
         include:
         # Set the minimum Go patch version for the given Go minor
         # Usable via ${{ matrix.GO_SEMVER }}
-        - go: '1.19'
-          GO_SEMVER: '~1.19.0'
+        - go: '1.24'
+          GO_SEMVER: '~1.24.1'
 
     runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: read
     continue-on-error: true
     steps:
+      - name: Harden the runner (Audit all outbound calls)
+        uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
+        with:
+          egress-policy: audit
+
+      - name: Checkout code
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+
       - name: Install Go
-        uses: actions/setup-go@v3
+        uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
         with:
           go-version: ${{ matrix.GO_SEMVER }}
           check-latest: true
@@ -43,32 +72,13 @@ jobs:
           printf "\n\nSystem environment:\n\n"
           env
 
-      - name: Cache the build cache
-        uses: actions/cache@v2
-        with:
-          # In order:
-          # * Module download cache
-          # * Build cache (Linux)
-          path: |
-            ~/go/pkg/mod
-            ~/.cache/go-build
-          key: cross-build-go${{ matrix.go }}-${{ matrix.goos }}-${{ hashFiles('**/go.sum') }}
-          restore-keys: |
-            cross-build-go${{ matrix.go }}-${{ matrix.goos }}
-
-      - name: Checkout code into the Go module directory
-        uses: actions/checkout@v3
-
       - name: Run Build
         env:
           CGO_ENABLED: 0
           GOOS: ${{ matrix.goos }}
+          GOARCH: ${{ matrix.goos == 'aix' && 'ppc64' || 'amd64' }}
         shell: bash
         continue-on-error: true
         working-directory: ./cmd/caddy
         run: |
-          GOOS=$GOOS go build -trimpath -o caddy-"$GOOS"-amd64 2> /dev/null
-          if [ $? -ne 0 ]; then
-            echo "::warning ::$GOOS Build Failed"
-            exit 0
-          fi
+          GOOS=$GOOS GOARCH=$GOARCH go build -tags=nobadger,nomysql,nopgx -trimpath -o caddy-"$GOOS"-$GOARCH 2> /dev/null

.github/workflows/lint.yml

@@ -10,26 +10,96 @@ on:
       - master
       - 2.*
 
+permissions:
+  contents: read
+
+env:
+  # https://github.com/actions/setup-go/issues/491
+  GOTOOLCHAIN: local
+
 jobs:
   # From https://github.com/golangci/golangci-lint-action
   golangci:
+    permissions:
+      contents: read # for actions/checkout to fetch code
+      pull-requests: read # for golangci/golangci-lint-action to fetch pull requests
     name: lint
     strategy:
       matrix:
-        os: [ubuntu-latest, macos-latest, windows-latest]
-    runs-on: ${{ matrix.os }}
+        os:
+          - linux
+          - mac
+          - windows
+
+        include:
+        - os: linux
+          OS_LABEL: ubuntu-latest
+
+        - os: mac
+          OS_LABEL: macos-14
+
+        - os: windows
+          OS_LABEL: windows-latest
+
+    runs-on: ${{ matrix.OS_LABEL }}
+
     steps:
-      - uses: actions/checkout@v3
-      - uses: actions/setup-go@v3
+      - name: Harden the runner (Audit all outbound calls)
+        uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
         with:
-          go-version: '~1.18.4'
+          egress-policy: audit
+
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+        with:
+          go-version: '~1.24'
           check-latest: true
 
       - name: golangci-lint
-        uses: golangci/golangci-lint-action@v3
+        uses: golangci/golangci-lint-action@4afd733a84b1f43292c63897423277bb7f4313a9 # v8.0.0
         with:
-          version: v1.47
+          version: latest
+
           # Windows times out frequently after about 5m50s if we don't set a longer timeout.
           args: --timeout 10m
+
           # Optional: show only new issues if it's a pull request. The default value is `false`.
           # only-new-issues: true
+
+  govulncheck:
+    permissions:
+      contents: read
+      pull-requests: read
+    runs-on: ubuntu-latest
+    steps:
+      - name: Harden the runner (Audit all outbound calls)
+        uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
+        with:
+          egress-policy: audit
+
+      - name: govulncheck
+        uses: golang/govulncheck-action@b625fbe08f3bccbe446d94fbf87fcc875a4f50ee # v1.0.4
+        with:
+          go-version-input: '~1.24.1'
+          check-latest: true
+  
+  dependency-review:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      pull-requests: write
+    steps:
+      - name: Harden the runner (Audit all outbound calls)
+        uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
+        with:
+          egress-policy: audit
+
+      - name: 'Checkout Repository'
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      - name: 'Dependency Review'
+        uses: actions/dependency-review-action@da24556b548a50705dd671f47852072ea4c105d9 # v4.7.1
+        with:
+          comment-summary-in-pr: on-failure
+          # https://github.com/actions/dependency-review-action/issues/430#issuecomment-1468975566
+          base-ref: ${{ github.event.pull_request.base.sha || 'master' }}
+          head-ref: ${{ github.event.pull_request.head.sha || github.ref }}

.github/workflows/release.yml

@@ -5,19 +5,28 @@ on:
     tags:
       - 'v*.*.*'
 
+env:
+  # https://github.com/actions/setup-go/issues/491
+  GOTOOLCHAIN: local
+
+permissions:
+  contents: read
+
 jobs:
   release:
     name: Release
     strategy:
       matrix:
-        os: [ ubuntu-latest ]
-        go: [ '1.19' ]
+        os: 
+          - ubuntu-latest
+        go: 
+          - '1.24'
 
         include:
         # Set the minimum Go patch version for the given Go minor
         # Usable via ${{ matrix.GO_SEMVER }}
-        - go: '1.19'
-          GO_SEMVER: '~1.19.0'
+        - go: '1.24'
+          GO_SEMVER: '~1.24.1'
 
     runs-on: ${{ matrix.os }}
     # https://github.com/sigstore/cosign/issues/1258#issuecomment-1002251233
@@ -29,19 +38,24 @@ jobs:
       contents: write
 
     steps:
+    - name: Harden the runner (Audit all outbound calls)
+      uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
+      with:
+        egress-policy: audit
+
+    - name: Checkout code
+      uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+      with:
+        fetch-depth: 0
+
     - name: Install Go
-      uses: actions/setup-go@v3
+      uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
       with:
         go-version: ${{ matrix.GO_SEMVER }}
         check-latest: true
 
-    - name: Checkout code
-      uses: actions/checkout@v3
-      with:
-        fetch-depth: 0
-
     # Force fetch upstream tags -- because 65 minutes
-    # tl;dr: actions/checkout@v3 runs this line:
+    # tl;dr: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 runs this line:
     #   git -c protocol.version=2 fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +ebc278ec98bb24f2852b61fde2a9bf2e3d83818b:refs/tags/
     # which makes its own local lightweight tag, losing all the annotations in the process. Our earlier script ran:
     #   git fetch --prune --unshallow
@@ -61,8 +75,8 @@ jobs:
         go env
         printf "\n\nSystem environment:\n\n"
         env
-        echo "::set-output name=version_tag::${GITHUB_REF/refs\/tags\//}"
-        echo "::set-output name=short_sha::$(git rev-parse --short HEAD)"
+        echo "version_tag=${GITHUB_REF/refs\/tags\//}" >> $GITHUB_OUTPUT
+        echo "short_sha=$(git rev-parse --short HEAD)" >> $GITHUB_OUTPUT
 
         # Add "pip install" CLI tools to PATH
         echo ~/.local/bin >> $GITHUB_PATH
@@ -74,10 +88,10 @@ jobs:
         TAG_MINOR=`echo ${TAG#v} | sed -e "s#$SEMVER_RE#\2#"`
         TAG_PATCH=`echo ${TAG#v} | sed -e "s#$SEMVER_RE#\3#"`
         TAG_SPECIAL=`echo ${TAG#v} | sed -e "s#$SEMVER_RE#\4#"`
-        echo "::set-output name=tag_major::${TAG_MAJOR}"
-        echo "::set-output name=tag_minor::${TAG_MINOR}"
-        echo "::set-output name=tag_patch::${TAG_PATCH}"
-        echo "::set-output name=tag_special::${TAG_SPECIAL}"
+        echo "tag_major=${TAG_MAJOR}" >> $GITHUB_OUTPUT
+        echo "tag_minor=${TAG_MINOR}" >> $GITHUB_OUTPUT
+        echo "tag_patch=${TAG_PATCH}" >> $GITHUB_OUTPUT
+        echo "tag_special=${TAG_SPECIAL}" >> $GITHUB_OUTPUT
 
     # Cloudsmith CLI tooling for pushing releases
     # See https://help.cloudsmith.io/docs/cli
@@ -94,32 +108,24 @@ jobs:
         # tags are only accepted if signed by Matt's key
         git verify-tag "${{ steps.vars.outputs.version_tag }}" || exit 1
 
-    - name: Cache the build cache
-      uses: actions/cache@v2
-      with:
-        # In order:
-        # * Module download cache
-        # * Build cache (Linux)
-        path: |
-          ~/go/pkg/mod
-          ~/.cache/go-build
-        key: ${{ runner.os }}-go${{ matrix.go }}-release-${{ hashFiles('**/go.sum') }}
-        restore-keys: |
-          ${{ runner.os }}-go${{ matrix.go }}-release
     - name: Install Cosign
-      uses: sigstore/cosign-installer@main
+      uses: sigstore/cosign-installer@e9a05e6d32d7ed22b5656cd874ef31af58d05bfa # main
     - name: Cosign version
       run: cosign version
     - name: Install Syft
-      uses: anchore/sbom-action/download-syft@main
+      uses: anchore/sbom-action/download-syft@9246b90769f852b3a8921f330c59e0b3f439d6e9 # main
     - name: Syft version
       run: syft version
+    - name: Install xcaddy
+      run: |
+        go install github.com/caddyserver/xcaddy/cmd/xcaddy@latest
+        xcaddy version
     # GoReleaser will take care of publishing those artifacts into the release
     - name: Run GoReleaser
-      uses: goreleaser/goreleaser-action@v2
+      uses: goreleaser/goreleaser-action@9c156ee8a17a598857849441385a2041ef570552 # v6.3.0
       with:
         version: latest
-        args: release --rm-dist
+        args: release --clean --timeout 60m
       env:
         GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         TAG: ${{ steps.vars.outputs.version_tag }}

.github/workflows/release_published.yml

@@ -5,19 +5,31 @@ on:
   release:
     types: [published]
 
+permissions:
+  contents: read
+
 jobs:
   release:
     name: Release Published
     strategy:
       matrix:
-        os: [ ubuntu-latest ]
+        os: 
+          - ubuntu-latest
     runs-on: ${{ matrix.os }}
-
+    permissions:
+      contents: read
+      pull-requests: read
+      actions: write
     steps:
 
     # See https://github.com/peter-evans/repository-dispatch
+    - name: Harden the runner (Audit all outbound calls)
+      uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
+      with:
+        egress-policy: audit
+
     - name: Trigger event on caddyserver/dist
-      uses: peter-evans/repository-dispatch@v1
+      uses: peter-evans/repository-dispatch@ff45666b9427631e3450c54a1bcbee4d9ff4d7c0 # v3.0.0
       with:
         token: ${{ secrets.REPO_DISPATCH_TOKEN }}
         repository: caddyserver/dist
@@ -25,7 +37,7 @@ jobs:
         client-payload: '{"tag": "${{ github.event.release.tag_name }}"}'
 
     - name: Trigger event on caddyserver/caddy-docker
-      uses: peter-evans/repository-dispatch@v1
+      uses: peter-evans/repository-dispatch@ff45666b9427631e3450c54a1bcbee4d9ff4d7c0 # v3.0.0
       with:
         token: ${{ secrets.REPO_DISPATCH_TOKEN }}
         repository: caddyserver/caddy-docker

.github/workflows/scorecard.yml

@@ -0,0 +1,86 @@
+# This workflow uses actions that are not certified by GitHub. They are provided
+# by a third-party and are governed by separate terms of service, privacy
+# policy, and support documentation.
+
+name: OpenSSF Scorecard supply-chain security
+on:
+  # For Branch-Protection check. Only the default branch is supported. See
+  # https://github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection
+  branch_protection_rule:
+  # To guarantee Maintained check is occasionally updated. See
+  # https://github.com/ossf/scorecard/blob/main/docs/checks.md#maintained
+  schedule:
+    - cron: '20 2 * * 5'
+  push:
+    branches: [ "master", "2.*" ]
+  pull_request:
+    branches: [ "master", "2.*" ]
+
+
+# Declare default permissions as read only.
+permissions: read-all
+
+jobs:
+  analysis:
+    name: Scorecard analysis
+    runs-on: ubuntu-latest
+    # `publish_results: true` only works when run from the default branch. conditional can be removed if disabled.
+    if: github.event.repository.default_branch == github.ref_name || github.event_name == 'pull_request'
+    permissions:
+      # Needed to upload the results to code-scanning dashboard.
+      security-events: write
+      # Needed to publish results and get a badge (see publish_results below).
+      id-token: write
+      # Uncomment the permissions below if installing in a private repository.
+      # contents: read
+      # actions: read
+
+    steps:
+      - name: Harden the runner (Audit all outbound calls)
+        uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
+        with:
+          egress-policy: audit
+
+      - name: "Checkout code"
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+        with:
+          persist-credentials: false
+
+      - name: "Run analysis"
+        uses: ossf/scorecard-action@f49aabe0b5af0936a0987cfb85d86b75731b0186 # v2.4.1
+        with:
+          results_file: results.sarif
+          results_format: sarif
+          # (Optional) "write" PAT token. Uncomment the `repo_token` line below if:
+          # - you want to enable the Branch-Protection check on a *public* repository, or
+          # - you are installing Scorecard on a *private* repository
+          # To create the PAT, follow the steps in https://github.com/ossf/scorecard-action?tab=readme-ov-file#authentication-with-fine-grained-pat-optional.
+          # repo_token: ${{ secrets.SCORECARD_TOKEN }}
+
+          # Public repositories:
+          #   - Publish results to OpenSSF REST API for easy access by consumers
+          #   - Allows the repository to include the Scorecard badge.
+          #   - See https://github.com/ossf/scorecard-action#publishing-results.
+          # For private repositories:
+          #   - `publish_results` will always be set to `false`, regardless
+          #     of the value entered here.
+          publish_results: true
+
+          # (Optional) Uncomment file_mode if you have a .gitattributes with files marked export-ignore
+          # file_mode: git
+
+      # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
+      # format to the repository Actions tab.
+      - name: "Upload artifact"
+        uses: actions/upload-artifact@4cec3d8aa04e39d1a68397de0c4cd6fb9dce8ec1 # v4.6.1
+        with:
+          name: SARIF file
+          path: results.sarif
+          retention-days: 5
+
+      # Upload the results to GitHub's code scanning dashboard (optional).
+      # Commenting out will disable upload of results to your repo's Code Scanning dashboard
+      - name: "Upload to code-scanning"
+        uses: github/codeql-action/upload-sarif@ce28f5bb42b7a9f2c824e633a3f6ee835bab6858 # v3.29.0
+        with:
+          sarif_file: results.sarif

.gitignore

@@ -3,6 +3,7 @@ _gitignore/
 Caddyfile
 Caddyfile.*
 !caddyfile/
+!caddyfile.go
 
 # artifacts from pprof tooling
 *.prof
@@ -11,6 +12,8 @@ Caddyfile.*
 # build artifacts and helpers
 cmd/caddy/caddy
 cmd/caddy/caddy.exe
+cmd/caddy/tmp/*.exe
+cmd/caddy/.env
 
 # mac specific
 .DS_Store

.golangci.yml

@@ -1,98 +1,118 @@
-linters-settings:
-  errcheck:
-    ignore: fmt:.*,go.uber.org/zap/zapcore:^Add.*
-    ignoretests: true
-
+version: "2"
+run:
+  issues-exit-code: 1
+  tests: false
+output:
+  formats:
+    text:
+      path: stdout
+      print-linter-name: true
+      print-issued-lines: true
 linters:
-  disable-all: true
+  default: none
   enable:
+    - asasalint
+    - asciicheck
+    - bidichk
     - bodyclose
-    - deadcode
+    - decorder
+    - dogsled
+    - dupl
+    - dupword
+    - durationcheck
     - errcheck
-    - gofmt
-    - goimports
+    - errname
+    - exhaustive
     - gosec
-    - gosimple
     - govet
+    - importas
     - ineffassign
     - misspell
     - prealloc
+    - promlinter
+    - sloglint
+    - sqlclosecheck
     - staticcheck
-    - structcheck
-    - typecheck
+    - testableexamples
+    - testifylint
+    - tparallel
     - unconvert
     - unused
-    - varcheck
-  # these are implicitly disabled:
-  # - asciicheck
-  # - depguard
-  # - dogsled
-  # - dupl
-  # - exhaustive
-  # - exportloopref
-  # - funlen
-  # - gci
-  # - gochecknoglobals
-  # - gochecknoinits
-  # - gocognit
-  # - goconst
-  # - gocritic
-  # - gocyclo
-  # - godot
-  # - godox
-  # - goerr113
-  # - gofumpt
-  # - goheader
-  # - golint
-  # - gomnd
-  # - gomodguard
-  # - goprintffuncname
-  # - interfacer
-  # - lll
-  # - maligned
-  # - nakedret
-  # - nestif
-  # - nlreturn
-  # - noctx
-  # - nolintlint
-  # - rowserrcheck
-  # - scopelint
-  # - sqlclosecheck
-  # - stylecheck
-  # - testpackage
-  # - unparam
-  # - whitespace
-  # - wsl
-
-run:
-  # default concurrency is a available CPU number.
-  # concurrency: 4 # explicitly omit this value to fully utilize available resources.
-  deadline: 5m
-  issues-exit-code: 1
-  tests: false
-
-# output configuration options
-output:
-  format: 'colored-line-number'
-  print-issued-lines: true
-  print-linter-name: true
-
-issues:
-  exclude-rules:
-    # we aren't calling unknown URL
-    - text: "G107" # G107: Url provided to HTTP request as taint input
-      linters:
-        - gosec
-    # as a web server that's expected to handle any template, this is totally in the hands of the user.
-    - text: "G203" # G203: Use of unescaped data in HTML templates
-      linters:
-        - gosec
-    # we're shelling out to known commands, not relying on user-defined input.
-    - text: "G204" # G204: Audit use of command execution
-      linters:
-        - gosec
-    # the choice of weakrand is deliberate, hence the named import "weakrand"
-    - path: modules/caddyhttp/reverseproxy/selectionpolicies.go
-      text: "G404" # G404: Insecure random number source (rand)
-      linters:
-        - gosec
+    - wastedassign
+    - whitespace
+    - zerologlint
+  settings:
+    staticcheck:
+      checks: ["all", "-ST1000", "-ST1003", "-ST1016", "-ST1020", "-ST1021", "-ST1022",  "-QF1006", "-QF1008"] # default, and exclude 1 more undesired check
+    errcheck:
+      exclude-functions:
+        - fmt.*
+        - (go.uber.org/zap/zapcore.ObjectEncoder).AddObject
+        - (go.uber.org/zap/zapcore.ObjectEncoder).AddArray
+    exhaustive:
+      ignore-enum-types: reflect.Kind|svc.Cmd
+  exclusions:
+    generated: lax
+    presets:
+      - comments
+      - common-false-positives
+      - legacy
+      - std-error-handling
+    rules:
+      - linters:
+          - gosec
+        text: G115 # TODO: Either we should fix the issues or nuke the linter if it's bad
+      - linters:
+          - gosec
+        text: G107 # we aren't calling unknown URL
+      - linters:
+          - gosec
+        text: G203 # as a web server that's expected to handle any template, this is totally in the hands of the user.
+      - linters:
+          - gosec
+        text: G204 # we're shelling out to known commands, not relying on user-defined input.
+      - linters:
+          - gosec
+        # the choice of weakrand is deliberate, hence the named import "weakrand"
+        path: modules/caddyhttp/reverseproxy/selectionpolicies.go
+        text: G404
+      - linters:
+          - gosec
+        path: modules/caddyhttp/reverseproxy/streaming.go
+        text: G404
+      - linters:
+          - dupl
+        path: modules/logging/filters.go
+      - linters:
+          - dupl
+        path: modules/caddyhttp/matchers.go
+      - linters:
+          - dupl
+        path: modules/caddyhttp/vars.go
+      - linters:
+          - errcheck
+        path: _test\.go
+    paths:
+      - third_party$
+      - builtin$
+      - examples$
+formatters:
+  enable:
+    - gci
+    - gofmt
+    - gofumpt
+    - goimports
+  settings:
+    gci:
+      sections:
+        - standard # Standard section: captures all standard packages.
+        - default # Default section: contains all imports that could not be matched to another section type.
+        - prefix(github.com/caddyserver/caddy/v2/cmd) # ensure that this is always at the top and always has a line break.
+        - prefix(github.com/caddyserver/caddy) # Custom section: groups all imports with the specified Prefix.
+      custom-order: true
+  exclusions:
+    generated: lax
+    paths:
+      - third_party$
+      - builtin$
+      - examples$

.goreleaser.yml

@@ -1,18 +1,28 @@
+version: 2
+
 before:
   hooks:
     # The build is done in this particular way to build Caddy in a designated directory named in .gitignore.
     # This is so we can run goreleaser on tag without Git complaining of being dirty. The main.go in cmd/caddy directory 
     # cannot be built within that directory due to changes necessary for the build causing Git to be dirty, which
     # subsequently causes gorleaser to refuse running.
+    - rm -rf caddy-build caddy-dist vendor
+    # vendor Caddy deps
+    - go mod vendor
     - mkdir -p caddy-build
     - cp cmd/caddy/main.go caddy-build/main.go
     - /bin/sh -c 'cd ./caddy-build && go mod init caddy'
+    # prepare syso files for windows embedding
+    - /bin/sh -c 'for a in amd64 arm arm64; do XCADDY_SKIP_BUILD=1 GOOS=windows GOARCH=$a xcaddy build {{.Env.TAG}}; done'
+    - /bin/sh -c 'mv /tmp/buildenv_*/*.syso caddy-build'
     # GoReleaser doesn't seem to offer {{.Tag}} at this stage, so we have to embed it into the env
     # so we run: TAG=$(git describe --abbrev=0) goreleaser release --rm-dist --skip-publish --skip-validate
     - go mod edit -require=github.com/caddyserver/caddy/v2@{{.Env.TAG}} ./caddy-build/go.mod
     # as of Go 1.16, `go` commands no longer automatically change go.{mod,sum}. We now have to explicitly
     # run `go mod tidy`. The `/bin/sh -c '...'` is because goreleaser can't find cd in PATH without shell invocation.
     - /bin/sh -c 'cd ./caddy-build && go mod tidy'
+    # vendor the deps of the prepared to-build module
+    - /bin/sh -c 'cd ./caddy-build && go mod vendor'
     - git clone --depth 1 https://github.com/caddyserver/dist caddy-dist
     - mkdir -p caddy-dist/man
     - go mod download
@@ -24,7 +34,6 @@ builds:
 - env:
   - CGO_ENABLED=0
   - GO111MODULE=on
-  main: main.go
   dir: ./caddy-build
   binary: caddy
   goos:
@@ -38,6 +47,7 @@ builds:
   - arm64
   - s390x
   - ppc64le
+  - riscv64
   goarm:
   - "5"
   - "6"
@@ -49,14 +59,20 @@ builds:
       goarch: ppc64le
     - goos: darwin
       goarch: s390x
+    - goos: darwin
+      goarch: riscv64
     - goos: windows
       goarch: ppc64le
     - goos: windows
       goarch: s390x
+    - goos: windows
+      goarch: riscv64
     - goos: freebsd
       goarch: ppc64le
     - goos: freebsd
       goarch: s390x
+    - goos: freebsd
+      goarch: riscv64
     - goos: freebsd
       goarch: arm
       goarm: "5"
@@ -65,24 +81,73 @@ builds:
   - -mod=readonly
   ldflags:
   - -s -w
+  tags:
+  - nobadger
+  - nomysql
+  - nopgx
+
 signs:
   - cmd: cosign
     signature: "${artifact}.sig"
     certificate: '{{ trimsuffix (trimsuffix .Env.artifact ".zip") ".tar.gz" }}.pem'
-    args: ["sign-blob", "--output-signature=${signature}", "--output-certificate", "${certificate}", "${artifact}"]
+    args: ["sign-blob", "--yes", "--output-signature=${signature}", "--output-certificate", "${certificate}", "${artifact}"]
     artifacts: all
+
 sboms:
   - artifacts: binary
     documents:
-    - '{{ .Binary }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}{{if .Arm}}v{{ .Arm }}{{end}}.sbom'
+      - >-
+        {{ .ProjectName }}_
+        {{- .Version }}_
+        {{- if eq .Os "darwin" }}mac{{ else }}{{ .Os }}{{ end }}_
+        {{- .Arch }}
+        {{- with .Arm }}v{{ . }}{{ end }}
+        {{- with .Mips }}_{{ . }}{{ end }}
+        {{- if not (eq .Amd64 "v1") }}{{ .Amd64 }}{{ end }}.sbom
     cmd: syft
     args: ["$artifact", "--file", "${document}", "--output", "cyclonedx-json"]
+
 archives:
-  - format_overrides:
+  - id: default
+    format_overrides:
       - goos: windows
-        format: zip
-    replacements:
-      darwin: mac
+        formats: zip
+    name_template: >-
+      {{ .ProjectName }}_
+      {{- .Version }}_
+      {{- if eq .Os "darwin" }}mac{{ else }}{{ .Os }}{{ end }}_
+      {{- .Arch }}
+      {{- with .Arm }}v{{ . }}{{ end }}
+      {{- with .Mips }}_{{ . }}{{ end }}
+      {{- if not (eq .Amd64 "v1") }}{{ .Amd64 }}{{ end }}
+  
+  # package the 'caddy-build' directory into a tarball,
+  # allowing users to build the exact same set of files as ours.
+  - id: source
+    meta: true
+    name_template: "{{ .ProjectName }}_{{ .Version }}_buildable-artifact"
+    files:
+      - src: LICENSE
+        dst: ./LICENSE
+      - src: README.md
+        dst: ./README.md
+      - src: AUTHORS
+        dst: ./AUTHORS
+      - src: ./caddy-build
+        dst: ./
+
+source:
+  enabled: true
+  name_template: '{{ .ProjectName }}_{{ .Version }}_src'
+  format: 'tar.gz'
+
+  # Additional files/template/globs you want to add to the source archive.
+  #
+  # Default: empty.
+  files:
+    - vendor
+
+
 checksum:
   algorithm: sha512
 
@@ -127,6 +192,8 @@ nfpms:
       preremove: ./caddy-dist/scripts/preremove.sh
       postremove: ./caddy-dist/scripts/postremove.sh
 
+    provides:
+      - httpd
 
 release:
   github:

.pre-commit-config.yaml

@@ -0,0 +1,20 @@
+repos:
+- repo: https://github.com/gitleaks/gitleaks
+  rev: v8.16.3
+  hooks:
+  - id: gitleaks
+- repo: https://github.com/golangci/golangci-lint
+  rev: v1.52.2
+  hooks:
+  - id: golangci-lint-config-verify
+  - id: golangci-lint
+  - id: golangci-lint-fmt
+- repo: https://github.com/jumanjihouse/pre-commit-hooks
+  rev: 3.0.0
+  hooks:
+  - id: shellcheck
+- repo: https://github.com/pre-commit/pre-commit-hooks
+  rev: v4.4.0
+  hooks:
+  - id: end-of-file-fixer
+  - id: trailing-whitespace

README.md

@@ -1,16 +1,23 @@
 <p align="center">
-	<a href="https://caddyserver.com"><img src="https://user-images.githubusercontent.com/1128849/36338535-05fb646a-136f-11e8-987b-e6901e717d5a.png" alt="Caddy" width="450"></a>
+	<a href="https://caddyserver.com">
+		<picture>
+			<source media="(prefers-color-scheme: dark)" srcset="https://user-images.githubusercontent.com/1128849/210187358-e2c39003-9a5e-4dd5-a783-6deb6483ee72.svg">
+			<source media="(prefers-color-scheme: light)" srcset="https://user-images.githubusercontent.com/1128849/210187356-dfb7f1c5-ac2e-43aa-bb23-fc014280ae1f.svg">
+			<img src="https://user-images.githubusercontent.com/1128849/210187356-dfb7f1c5-ac2e-43aa-bb23-fc014280ae1f.svg" alt="Caddy" width="550">
+		</picture>
+	</a>
 	<br>
-	<h3 align="center">a <a href="https://zerossl.com"><img src="https://caddyserver.com/resources/images/zerossl-logo.svg" height="28" valign="middle"></a> project</h3>
+	<h3 align="center">a <a href="https://zerossl.com"><img src="https://user-images.githubusercontent.com/55066419/208327323-2770dc16-ec09-43a0-9035-c5b872c2ad7f.svg" height="28" style="vertical-align: -7.7px" valign="middle"></a> project</h3>
 </p>
 <hr>
 <h3 align="center">Every site on HTTPS</h3>
 <p align="center">Caddy is an extensible server platform that uses TLS by default.</p>
 <p align="center">
-	<a href="https://github.com/caddyserver/caddy/actions?query=workflow%3ACross-Platform"><img src="https://github.com/caddyserver/caddy/workflows/Cross-Platform/badge.svg"></a>
+	<a href="https://github.com/caddyserver/caddy/actions/workflows/ci.yml"><img src="https://github.com/caddyserver/caddy/actions/workflows/ci.yml/badge.svg"></a>
+	<a href="https://www.bestpractices.dev/projects/7141"><img src="https://www.bestpractices.dev/projects/7141/badge"></a>
 	<a href="https://pkg.go.dev/github.com/caddyserver/caddy/v2"><img src="https://img.shields.io/badge/godoc-reference-%23007d9c.svg"></a>
 	<br>
-	<a href="https://twitter.com/caddyserver" title="@caddyserver on Twitter"><img src="https://img.shields.io/badge/twitter-@caddyserver-55acee.svg" alt="@caddyserver on Twitter"></a>
+	<a href="https://x.com/caddyserver" title="@caddyserver on Twitter"><img src="https://img.shields.io/twitter/follow/caddyserver" alt="@caddyserver on Twitter"></a>
 	<a href="https://caddy.community" title="Caddy Forum"><img src="https://img.shields.io/badge/community-forum-ff69b4.svg" alt="Caddy Forum"></a>
 	<br>
 	<a href="https://sourcegraph.com/github.com/caddyserver/caddy?badge" title="Caddy on Sourcegraph"><img src="https://sourcegraph.com/github.com/caddyserver/caddy/-/badge.svg" alt="Caddy on Sourcegraph"></a>
@@ -40,11 +47,17 @@
 <p align="center">
 	<b>Powered by</b>
 	<br>
-	<a href="https://github.com/caddyserver/certmagic"><img src="https://user-images.githubusercontent.com/1128849/49704830-49d37200-fbd5-11e8-8385-767e0cd033c3.png" alt="CertMagic" width="250"></a>
+	<a href="https://github.com/caddyserver/certmagic">
+		<picture>
+			<source media="(prefers-color-scheme: dark)" srcset="https://user-images.githubusercontent.com/55066419/206946718-740b6371-3df3-4d72-a822-47e4c48af999.png">
+			<source media="(prefers-color-scheme: light)" srcset="https://user-images.githubusercontent.com/1128849/49704830-49d37200-fbd5-11e8-8385-767e0cd033c3.png">
+			<img src="https://user-images.githubusercontent.com/1128849/49704830-49d37200-fbd5-11e8-8385-767e0cd033c3.png" alt="CertMagic" width="250">
+		</picture>
+	</a>
 </p>
 
 
-## [Features](https://caddyserver.com/v2)
+## [Features](https://caddyserver.com/features)
 
 - **Easy configuration** with the [Caddyfile](https://caddyserver.com/docs/caddyfile)
 - **Powerful configuration** with its [native JSON config](https://caddyserver.com/docs/json/)
@@ -55,15 +68,16 @@
 	- Fully-managed local CA for internal names & IPs
 	- Can coordinate with other Caddy instances in a cluster
 	- Multi-issuer fallback
+	- Encrypted ClientHello (ECH) support
 - **Stays up when other servers go down** due to TLS/OCSP/certificate-related issues
 - **Production-ready** after serving trillions of requests and managing millions of TLS certificates
 - **Scales to hundreds of thousands of sites** as proven in production
-- **HTTP/1.1, HTTP/2, and HTTP/3** supported all by default
+- **HTTP/1.1, HTTP/2, and HTTP/3** all supported by default
 - **Highly extensible** [modular architecture](https://caddyserver.com/docs/architecture) lets Caddy do anything without bloat
 - **Runs anywhere** with **no external dependencies** (not even libc)
 - Written in Go, a language with higher **memory safety guarantees** than other servers
 - Actually **fun to use**
-- So much more to [discover](https://caddyserver.com/v2)
+- So much more to [discover](https://caddyserver.com/features)
 
 ## Install
 
@@ -75,10 +89,10 @@ See [our online documentation](https://caddyserver.com/docs/install) for other i
 
 Requirements:
 
-- [Go 1.18 or newer](https://golang.org/dl/)
+- [Go 1.24.0 or newer](https://golang.org/dl/)
 
 ### For development
- 
+
 _**Note:** These steps [will not embed proper version information](https://github.com/golang/go/issues/29228). For that, please follow the instructions in the next section._
 
 ```bash
@@ -119,7 +133,7 @@ $ xcaddy build
 4. Initialize a Go module: `go mod init caddy`
 5. (Optional) Pin Caddy version: `go get github.com/caddyserver/caddy/v2@version` replacing `version` with a git tag, commit, or branch name.
 6. (Optional) Add plugins by adding their import: `_ "import/path/here"`
-7. Compile: `go build`
+7. Compile: `go build -tags=nobadger,nomysql,nopgx`
 
 
 
@@ -164,7 +178,7 @@ The docs are also open source. You can contribute to them here: https://github.c
 
 ## Getting help
 
-- We advise companies using Caddy to secure a support contract through [Ardan Labs](https://www.ardanlabs.com/my/contact-us?dd=caddy) before help is needed.
+- We advise companies using Caddy to secure a support contract through [Ardan Labs](https://www.ardanlabs.com) before help is needed.
 
 - A [sponsorship](https://github.com/sponsors/mholt) goes a long way! We can offer private help to sponsors. If Caddy is benefitting your company, please consider a sponsorship. This not only helps fund full-time work to ensure the longevity of the project, it provides your company the resources, support, and discounts you need; along with being a great look for your company to your customers and potential customers!
 
@@ -180,9 +194,9 @@ Matthew Holt began developing Caddy in 2014 while studying computer science at B
 
 **The name "Caddy" is trademarked.** The name of the software is "Caddy", not "Caddy Server" or "CaddyServer". Please call it "Caddy" or, if you wish to clarify, "the Caddy web server". Caddy is a registered trademark of Stack Holdings GmbH.
 
-- _Project on Twitter: [@caddyserver](https://twitter.com/caddyserver)_
-- _Author on Twitter: [@mholt6](https://twitter.com/mholt6)_
+- _Project on X: [@caddyserver](https://x.com/caddyserver)_
+- _Author on X: [@mholt6](https://x.com/mholt6)_
 
 Caddy is a project of [ZeroSSL](https://zerossl.com), a Stack Holdings company.
 
-Debian package repository hosting is graciously provided by [Cloudsmith](https://cloudsmith.com). Cloudsmith is the only fully hosted, cloud-native, universal package management solution, that enables your organization to create, store and share packages in any format, to any place, with total confidence.
+Debian package repository hosting is graciously provided by [Cloudsmith](https://cloudsmith.com). Cloudsmith is the only fully hosted, cloud-native, universal package management solution, that enables your organization to create, store and share packages in any format, to any place, with total confidence.

admin.go

@@ -26,7 +26,6 @@ import (
 	"expvar"
 	"fmt"
 	"hash"
-	"hash/fnv"
 	"io"
 	"net"
 	"net/http"
@@ -35,17 +34,30 @@ import (
 	"os"
 	"path"
 	"regexp"
+	"slices"
 	"strconv"
 	"strings"
 	"sync"
 	"time"
 
 	"github.com/caddyserver/certmagic"
+	"github.com/cespare/xxhash/v2"
 	"github.com/prometheus/client_golang/prometheus"
 	"go.uber.org/zap"
 	"go.uber.org/zap/zapcore"
 )
 
+func init() {
+	// The hard-coded default `DefaultAdminListen` can be overridden
+	// by setting the `CADDY_ADMIN` environment variable.
+	// The environment variable may be used by packagers to change
+	// the default admin address to something more appropriate for
+	// that platform. See #5317 for discussion.
+	if env, exists := os.LookupEnv("CADDY_ADMIN"); exists {
+		DefaultAdminListen = env
+	}
+}
+
 // AdminConfig configures Caddy's API endpoint, which is used
 // to manage Caddy while it is running.
 type AdminConfig struct {
@@ -57,7 +69,14 @@ type AdminConfig struct {
 
 	// The address to which the admin endpoint's listener should
 	// bind itself. Can be any single network address that can be
-	// parsed by Caddy. Default: localhost:2019
+	// parsed by Caddy. Accepts placeholders.
+	// Default: the value of the `CADDY_ADMIN` environment variable,
+	// or `localhost:2019` otherwise.
+	//
+	// Remember: When changing this value through a config reload,
+	// be sure to use the `--address` CLI flag to specify the current
+	// admin address if the currently-running admin endpoint is not
+	// the default address.
 	Listen string `json:"listen,omitempty"`
 
 	// If true, CORS headers will be emitted, and requests to the
@@ -156,7 +175,7 @@ type IdentityConfig struct {
 //
 // EXPERIMENTAL: Subject to change.
 type RemoteAdmin struct {
-	// The address on which to start the secure listener.
+	// The address on which to start the secure listener. Accepts placeholders.
 	// Default: :2021
 	Listen string `json:"listen,omitempty"`
 
@@ -195,14 +214,15 @@ type AdminPermissions struct {
 
 // newAdminHandler reads admin's config and returns an http.Handler suitable
 // for use in an admin endpoint server, which will be listening on listenAddr.
-func (admin *AdminConfig) newAdminHandler(addr NetworkAddress, remote bool) adminHandler {
+func (admin *AdminConfig) newAdminHandler(addr NetworkAddress, remote bool, _ Context) adminHandler {
 	muxWrap := adminHandler{mux: http.NewServeMux()}
 
 	// secure the local or remote endpoint respectively
 	if remote {
 		muxWrap.remoteControl = admin.Remote
 	} else {
-		muxWrap.enforceHost = !addr.isWildcardInterface()
+		// see comment in allowedOrigins() as to why we disable the host check for unix/fd networks
+		muxWrap.enforceHost = !addr.isWildcardInterface() && !addr.IsUnixNetwork() && !addr.IsFdNetwork()
 		muxWrap.allowedOrigins = admin.allowedOrigins(addr)
 		muxWrap.enforceOrigin = admin.EnforceOrigin
 	}
@@ -251,7 +271,6 @@ func (admin *AdminConfig) newAdminHandler(addr NetworkAddress, remote bool) admi
 	// register third-party module endpoints
 	for _, m := range GetModules("admin.api") {
 		router := m.New().(AdminRouter)
-		handlerLabel := m.ID.Name()
 		for _, route := range router.Routes() {
 			addRoute(route.Pattern, handlerLabel, route.Handler)
 		}
@@ -292,22 +311,43 @@ func (admin AdminConfig) allowedOrigins(addr NetworkAddress) []*url.URL {
 	for _, o := range admin.Origins {
 		uniqueOrigins[o] = struct{}{}
 	}
-	if admin.Origins == nil {
+	// RFC 2616, Section 14.26:
+	// "A client MUST include a Host header field in all HTTP/1.1 request
+	// messages. If the requested URI does not include an Internet host
+	// name for the service being requested, then the Host header field MUST
+	// be given with an empty value."
+	//
+	// UPDATE July 2023: Go broke this by patching a minor security bug in 1.20.6.
+	// Understandable, but frustrating. See:
+	// https://github.com/golang/go/issues/60374
+	// See also the discussion here:
+	// https://github.com/golang/go/issues/61431
+	//
+	// We can no longer conform to RFC 2616 Section 14.26 from either Go or curl
+	// in purity. (Curl allowed no host between 7.40 and 7.50, but now requires a
+	// bogus host; see https://superuser.com/a/925610.) If we disable Host/Origin
+	// security checks, the infosec community assures me that it is secure to do
+	// so, because:
+	//
+	// 1) Browsers do not allow access to unix sockets
+	// 2) DNS is irrelevant to unix sockets
+	//
+	// If either of those two statements ever fail to hold true, it is not the
+	// fault of Caddy.
+	//
+	// Thus, we do not fill out allowed origins and do not enforce Host
+	// requirements for unix sockets. Enforcing it leads to confusion and
+	// frustration, when UDS have their own permissions from the OS.
+	// Enforcing host requirements here is effectively security theater,
+	// and a false sense of security.
+	//
+	// See also the discussion in #6832.
+	if admin.Origins == nil && !addr.IsUnixNetwork() && !addr.IsFdNetwork() {
 		if addr.isLoopback() {
-			if addr.IsUnixNetwork() {
-				// RFC 2616, Section 14.26:
-				// "A client MUST include a Host header field in all HTTP/1.1 request
-				// messages. If the requested URI does not include an Internet host
-				// name for the service being requested, then the Host header field MUST
-				// be given with an empty value."
-				uniqueOrigins[""] = struct{}{}
-			} else {
-				uniqueOrigins[net.JoinHostPort("localhost", addr.port())] = struct{}{}
-				uniqueOrigins[net.JoinHostPort("::1", addr.port())] = struct{}{}
-				uniqueOrigins[net.JoinHostPort("127.0.0.1", addr.port())] = struct{}{}
-			}
-		}
-		if !addr.IsUnixNetwork() {
+			uniqueOrigins[net.JoinHostPort("localhost", addr.port())] = struct{}{}
+			uniqueOrigins[net.JoinHostPort("::1", addr.port())] = struct{}{}
+			uniqueOrigins[net.JoinHostPort("127.0.0.1", addr.port())] = struct{}{}
+		} else {
 			uniqueOrigins[addr.JoinHostPort(0)] = struct{}{}
 		}
 	}
@@ -338,7 +378,9 @@ func (admin AdminConfig) allowedOrigins(addr NetworkAddress) []*url.URL {
 // for the admin endpoint exists in cfg, a default one is used, so
 // that there is always an admin server (unless it is explicitly
 // configured to be disabled).
-func replaceLocalAdminServer(cfg *Config) error {
+// Critically note that some elements and functionality of the context
+// may not be ready, e.g. storage. Tread carefully.
+func replaceLocalAdminServer(cfg *Config, ctx Context) error {
 	// always* be sure to close down the old admin endpoint
 	// as gracefully as possible, even if the new one is
 	// disabled -- careful to use reference to the current
@@ -380,9 +422,16 @@ func replaceLocalAdminServer(cfg *Config) error {
 		return err
 	}
 
-	handler := cfg.Admin.newAdminHandler(addr, false)
+	handler := cfg.Admin.newAdminHandler(addr, false, ctx)
 
-	ln, err := Listen(addr.Network, addr.JoinHostPort(0))
+	// run the provisioners for loaded modules to make sure local
+	// state is properly re-initialized in the new admin server
+	err = cfg.Admin.provisionAdminRouters(ctx)
+	if err != nil {
+		return err
+	}
+
+	ln, err := addr.Listen(context.TODO(), 0, net.ListenConfig{})
 	if err != nil {
 		return err
 	}
@@ -403,7 +452,7 @@ func replaceLocalAdminServer(cfg *Config) error {
 		serverMu.Lock()
 		server := localAdminServer
 		serverMu.Unlock()
-		if err := server.Serve(ln); !errors.Is(err, http.ErrServerClosed) {
+		if err := server.Serve(ln.(net.Listener)); !errors.Is(err, http.ErrServerClosed) {
 			adminLogger.Error("admin server shutdown for unknown reason", zap.Error(err))
 		}
 	}()
@@ -431,7 +480,6 @@ func manageIdentity(ctx Context, cfg *Config) error {
 	// import the caddytls package -- but it works
 	if cfg.Admin.Identity.IssuersRaw == nil {
 		cfg.Admin.Identity.IssuersRaw = []json.RawMessage{
-			json.RawMessage(`{"module": "zerossl"}`),
 			json.RawMessage(`{"module": "acme"}`),
 		}
 	}
@@ -502,7 +550,14 @@ func replaceRemoteAdminServer(ctx Context, cfg *Config) error {
 
 	// make the HTTP handler but disable Host/Origin enforcement
 	// because we are using TLS authentication instead
-	handler := cfg.Admin.newAdminHandler(addr, true)
+	handler := cfg.Admin.newAdminHandler(addr, true, ctx)
+
+	// run the provisioners for loaded modules to make sure local
+	// state is properly re-initialized in the new admin server
+	err = cfg.Admin.provisionAdminRouters(ctx)
+	if err != nil {
+		return err
+	}
 
 	// create client certificate pool for TLS mutual auth, and extract public keys
 	// so that we can enforce access controls at the application layer
@@ -549,10 +604,11 @@ func replaceRemoteAdminServer(ctx Context, cfg *Config) error {
 	serverMu.Unlock()
 
 	// start listener
-	ln, err := Listen(addr.Network, addr.JoinHostPort(0))
+	lnAny, err := addr.Listen(ctx, 0, net.ListenConfig{})
 	if err != nil {
 		return err
 	}
+	ln := lnAny.(net.Listener)
 	ln = tls.NewListener(ln, tlsConfig)
 
 	go func() {
@@ -571,12 +627,13 @@ func replaceRemoteAdminServer(ctx Context, cfg *Config) error {
 }
 
 func (ident *IdentityConfig) certmagicConfig(logger *zap.Logger, makeCache bool) *certmagic.Config {
+	var cmCfg *certmagic.Config
 	if ident == nil {
 		// user might not have configured identity; that's OK, we can still make a
 		// certmagic config, although it'll be mostly useless for remote management
 		ident = new(IdentityConfig)
 	}
-	cmCfg := &certmagic.Config{
+	template := certmagic.Config{
 		Storage: DefaultStorage, // do not act as part of a cluster (this is for the server's local identity)
 		Logger:  logger,
 		Issuers: ident.issuers,
@@ -586,9 +643,11 @@ func (ident *IdentityConfig) certmagicConfig(logger *zap.Logger, makeCache bool)
 			GetConfigForCert: func(certmagic.Certificate) (*certmagic.Config, error) {
 				return cmCfg, nil
 			},
+			Logger: logger.Named("cache"),
 		})
 	}
-	return certmagic.New(identityCertCache, *cmCfg)
+	cmCfg = certmagic.New(identityCertCache, template)
+	return cmCfg
 }
 
 // IdentityCredentials returns this instance's configured, managed identity credentials
@@ -629,13 +688,7 @@ func (remote RemoteAdmin) enforceAccessControls(r *http.Request) error {
 					// key recognized; make sure its HTTP request is permitted
 					for _, accessPerm := range adminAccess.Permissions {
 						// verify method
-						methodFound := accessPerm.Methods == nil
-						for _, method := range accessPerm.Methods {
-							if method == r.Method {
-								methodFound = true
-								break
-							}
-						}
+						methodFound := accessPerm.Methods == nil || slices.Contains(accessPerm.Methods, r.Method)
 						if !methodFound {
 							return APIError{
 								HTTPStatus: http.StatusForbidden,
@@ -831,13 +884,9 @@ func (h adminHandler) handleError(w http.ResponseWriter, r *http.Request, err er
 // a trustworthy/expected value. This helps to mitigate DNS
 // rebinding attacks.
 func (h adminHandler) checkHost(r *http.Request) error {
-	var allowed bool
-	for _, allowedOrigin := range h.allowedOrigins {
-		if r.Host == allowedOrigin.Host {
-			allowed = true
-			break
-		}
-	}
+	allowed := slices.ContainsFunc(h.allowedOrigins, func(u *url.URL) bool {
+		return r.Host == u.Host
+	})
 	if !allowed {
 		return APIError{
 			HTTPStatus: http.StatusForbidden,
@@ -897,9 +946,9 @@ func (h adminHandler) originAllowed(origin *url.URL) bool {
 	return false
 }
 
-// etagHasher returns a the hasher we used on the config to both
+// etagHasher returns the hasher we used on the config to both
 // produce and verify ETags.
-func etagHasher() hash.Hash32 { return fnv.New32a() }
+func etagHasher() hash.Hash { return xxhash.New() }
 
 // makeEtag returns an Etag header value (including quotes) for
 // the given config path and hash of contents at that path.
@@ -907,17 +956,28 @@ func makeEtag(path string, hash hash.Hash) string {
 	return fmt.Sprintf(`"%s %x"`, path, hash.Sum(nil))
 }
 
+// This buffer pool is used to keep buffers for
+// reading the config file during eTag header generation
+var bufferPool = sync.Pool{
+	New: func() any {
+		return new(bytes.Buffer)
+	},
+}
+
 func handleConfig(w http.ResponseWriter, r *http.Request) error {
 	switch r.Method {
 	case http.MethodGet:
 		w.Header().Set("Content-Type", "application/json")
-		// Set the ETag as a trailer header.
-		// The alternative is to write the config to a buffer, and
-		// then hash that.
-		w.Header().Set("Trailer", "ETag")
-
 		hash := etagHasher()
-		configWriter := io.MultiWriter(w, hash)
+
+		// Read the config into a buffer instead of writing directly to
+		// the response writer, as we want to set the ETag as the header,
+		// not the trailer.
+		buf := bufferPool.Get().(*bytes.Buffer)
+		buf.Reset()
+		defer bufferPool.Put(buf)
+
+		configWriter := io.MultiWriter(buf, hash)
 		err := readConfig(r.URL.Path, configWriter)
 		if err != nil {
 			return APIError{HTTPStatus: http.StatusBadRequest, Err: err}
@@ -926,6 +986,10 @@ func handleConfig(w http.ResponseWriter, r *http.Request) error {
 		// we could consider setting up a sync.Pool for the summed
 		// hashes to reduce GC pressure.
 		w.Header().Set("Etag", makeEtag(r.URL.Path, hash))
+		_, err = w.Write(buf.Bytes())
+		if err != nil {
+			return APIError{HTTPStatus: http.StatusInternalServerError, Err: err}
+		}
 
 		return nil
 
@@ -994,9 +1058,9 @@ func handleConfigID(w http.ResponseWriter, r *http.Request) error {
 	id := parts[2]
 
 	// map the ID to the expanded path
-	currentCtxMu.RLock()
+	rawCfgMu.RLock()
 	expanded, ok := rawCfgIndex[id]
-	defer currentCtxMu.RUnlock()
+	rawCfgMu.RUnlock()
 	if !ok {
 		return APIError{
 			HTTPStatus: http.StatusNotFound,
@@ -1086,7 +1150,7 @@ traverseLoop:
 						return fmt.Errorf("[%s] invalid array index '%s': %v",
 							path, idxStr, err)
 					}
-					if idx < 0 || idx >= len(arr) {
+					if idx < 0 || (method != http.MethodPut && idx >= len(arr)) || idx > len(arr) {
 						return fmt.Errorf("[%s] array index out of bounds: %s", path, idxStr)
 					}
 				}
@@ -1149,15 +1213,27 @@ traverseLoop:
 					}
 				case http.MethodPut:
 					if _, ok := v[part]; ok {
-						return fmt.Errorf("[%s] key already exists: %s", path, part)
+						return APIError{
+							HTTPStatus: http.StatusConflict,
+							Err:        fmt.Errorf("[%s] key already exists: %s", path, part),
+						}
 					}
 					v[part] = val
 				case http.MethodPatch:
 					if _, ok := v[part]; !ok {
-						return fmt.Errorf("[%s] key does not exist: %s", path, part)
+						return APIError{
+							HTTPStatus: http.StatusNotFound,
+							Err:        fmt.Errorf("[%s] key does not exist: %s", path, part),
+						}
 					}
 					v[part] = val
 				case http.MethodDelete:
+					if _, ok := v[part]; !ok {
+						return APIError{
+							HTTPStatus: http.StatusNotFound,
+							Err:        fmt.Errorf("[%s] key does not exist: %s", path, part),
+						}
+					}
 					delete(v, part)
 				default:
 					return fmt.Errorf("unrecognized method %s", method)
@@ -1246,7 +1322,10 @@ func (e APIError) Error() string {
 // parseAdminListenAddr extracts a singular listen address from either addr
 // or defaultAddr, returning the network and the address of the listener.
 func parseAdminListenAddr(addr string, defaultAddr string) (NetworkAddress, error) {
-	input := addr
+	input, err := NewReplacer().ReplaceOrErr(addr, true, true)
+	if err != nil {
+		return NetworkAddress{}, fmt.Errorf("replacing listen address: %v", err)
+	}
 	if input == "" {
 		input = defaultAddr
 	}
@@ -1296,7 +1375,7 @@ var (
 // will get deleted before the process gracefully exits.
 func PIDFile(filename string) error {
 	pid := []byte(strconv.Itoa(os.Getpid()) + "\n")
-	err := os.WriteFile(filename, pid, 0600)
+	err := os.WriteFile(filename, pid, 0o600)
 	if err != nil {
 		return err
 	}

admin_test.go

@@ -15,12 +15,20 @@
 package caddy
 
 import (
+	"context"
+	"crypto/x509"
 	"encoding/json"
 	"fmt"
+	"maps"
 	"net/http"
+	"net/http/httptest"
 	"reflect"
 	"sync"
 	"testing"
+
+	"github.com/caddyserver/certmagic"
+	"github.com/prometheus/client_golang/prometheus"
+	dto "github.com/prometheus/client_model/go"
 )
 
 var testCfg = []byte(`{
@@ -75,6 +83,12 @@ func TestUnsyncedConfigAccess(t *testing.T) {
 			path:   "/bar/qq",
 			expect: `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c"]}`,
 		},
+		{
+			method:    "DELETE",
+			path:      "/bar/qq",
+			expect:    `{"foo": "jet", "bar": {"aa": "bb"}, "list": ["a", "b", "c"]}`,
+			shouldErr: true,
+		},
 		{
 			method:  "POST",
 			path:    "/list",
@@ -197,3 +211,719 @@ func BenchmarkLoad(b *testing.B) {
 		Load(testCfg, true)
 	}
 }
+
+func TestAdminHandlerErrorHandling(t *testing.T) {
+	initAdminMetrics()
+
+	handler := adminHandler{
+		mux: http.NewServeMux(),
+	}
+
+	handler.mux.Handle("/error", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		err := fmt.Errorf("test error")
+		handler.handleError(w, r, err)
+	}))
+
+	req := httptest.NewRequest(http.MethodGet, "/error", nil)
+	rr := httptest.NewRecorder()
+
+	handler.ServeHTTP(rr, req)
+
+	if rr.Code == http.StatusOK {
+		t.Error("expected error response, got success")
+	}
+
+	var apiErr APIError
+	if err := json.NewDecoder(rr.Body).Decode(&apiErr); err != nil {
+		t.Fatalf("decoding response: %v", err)
+	}
+	if apiErr.Message != "test error" {
+		t.Errorf("expected error message 'test error', got '%s'", apiErr.Message)
+	}
+}
+
+func initAdminMetrics() {
+	if adminMetrics.requestErrors != nil {
+		prometheus.Unregister(adminMetrics.requestErrors)
+	}
+	if adminMetrics.requestCount != nil {
+		prometheus.Unregister(adminMetrics.requestCount)
+	}
+
+	adminMetrics.requestErrors = prometheus.NewCounterVec(prometheus.CounterOpts{
+		Namespace: "caddy",
+		Subsystem: "admin_http",
+		Name:      "request_errors_total",
+		Help:      "Number of errors that occurred handling admin endpoint requests",
+	}, []string{"handler", "path", "method"})
+
+	adminMetrics.requestCount = prometheus.NewCounterVec(prometheus.CounterOpts{
+		Namespace: "caddy",
+		Subsystem: "admin_http",
+		Name:      "requests_total",
+		Help:      "Count of requests to the admin endpoint",
+	}, []string{"handler", "path", "code", "method"}) // Added code and method labels
+
+	prometheus.MustRegister(adminMetrics.requestErrors)
+	prometheus.MustRegister(adminMetrics.requestCount)
+}
+
+func TestAdminHandlerBuiltinRouteErrors(t *testing.T) {
+	initAdminMetrics()
+
+	cfg := &Config{
+		Admin: &AdminConfig{
+			Listen: "localhost:2019",
+		},
+	}
+
+	err := replaceLocalAdminServer(cfg, Context{})
+	if err != nil {
+		t.Fatalf("setting up admin server: %v", err)
+	}
+	defer func() {
+		stopAdminServer(localAdminServer)
+	}()
+
+	tests := []struct {
+		name           string
+		path           string
+		method         string
+		expectedStatus int
+	}{
+		{
+			name:           "stop endpoint wrong method",
+			path:           "/stop",
+			method:         http.MethodGet,
+			expectedStatus: http.StatusMethodNotAllowed,
+		},
+		{
+			name:           "config endpoint wrong content-type",
+			path:           "/config/",
+			method:         http.MethodPost,
+			expectedStatus: http.StatusBadRequest,
+		},
+		{
+			name:           "config ID missing ID",
+			path:           "/id/",
+			method:         http.MethodGet,
+			expectedStatus: http.StatusBadRequest,
+		},
+	}
+
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			req := httptest.NewRequest(test.method, fmt.Sprintf("http://localhost:2019%s", test.path), nil)
+			rr := httptest.NewRecorder()
+
+			localAdminServer.Handler.ServeHTTP(rr, req)
+
+			if rr.Code != test.expectedStatus {
+				t.Errorf("expected status %d but got %d", test.expectedStatus, rr.Code)
+			}
+
+			metricValue := testGetMetricValue(map[string]string{
+				"path":    test.path,
+				"handler": "admin",
+				"method":  test.method,
+			})
+			if metricValue != 1 {
+				t.Errorf("expected error metric to be incremented once, got %v", metricValue)
+			}
+		})
+	}
+}
+
+func testGetMetricValue(labels map[string]string) float64 {
+	promLabels := prometheus.Labels{}
+	maps.Copy(promLabels, labels)
+
+	metric, err := adminMetrics.requestErrors.GetMetricWith(promLabels)
+	if err != nil {
+		return 0
+	}
+
+	pb := &dto.Metric{}
+	metric.Write(pb)
+	return pb.GetCounter().GetValue()
+}
+
+type mockRouter struct {
+	routes []AdminRoute
+}
+
+func (m mockRouter) Routes() []AdminRoute {
+	return m.routes
+}
+
+type mockModule struct {
+	mockRouter
+}
+
+func (m *mockModule) CaddyModule() ModuleInfo {
+	return ModuleInfo{
+		ID: "admin.api.mock",
+		New: func() Module {
+			mm := &mockModule{
+				mockRouter: mockRouter{
+					routes: m.routes,
+				},
+			}
+			return mm
+		},
+	}
+}
+
+func TestNewAdminHandlerRouterRegistration(t *testing.T) {
+	originalModules := make(map[string]ModuleInfo)
+	maps.Copy(originalModules, modules)
+	defer func() {
+		modules = originalModules
+	}()
+
+	mockRoute := AdminRoute{
+		Pattern: "/mock",
+		Handler: AdminHandlerFunc(func(w http.ResponseWriter, r *http.Request) error {
+			w.WriteHeader(http.StatusOK)
+			return nil
+		}),
+	}
+
+	mock := &mockModule{
+		mockRouter: mockRouter{
+			routes: []AdminRoute{mockRoute},
+		},
+	}
+	RegisterModule(mock)
+
+	addr, err := ParseNetworkAddress("localhost:2019")
+	if err != nil {
+		t.Fatalf("Failed to parse address: %v", err)
+	}
+
+	admin := &AdminConfig{
+		EnforceOrigin: false,
+	}
+	handler := admin.newAdminHandler(addr, false, Context{})
+
+	req := httptest.NewRequest("GET", "/mock", nil)
+	req.Host = "localhost:2019"
+	rr := httptest.NewRecorder()
+
+	handler.ServeHTTP(rr, req)
+
+	if rr.Code != http.StatusOK {
+		t.Errorf("Expected status code %d but got %d", http.StatusOK, rr.Code)
+		t.Logf("Response body: %s", rr.Body.String())
+	}
+
+	if len(admin.routers) != 1 {
+		t.Errorf("Expected 1 router to be stored, got %d", len(admin.routers))
+	}
+}
+
+type mockProvisionableRouter struct {
+	mockRouter
+	provisionErr error
+	provisioned  bool
+}
+
+func (m *mockProvisionableRouter) Provision(Context) error {
+	m.provisioned = true
+	return m.provisionErr
+}
+
+type mockProvisionableModule struct {
+	*mockProvisionableRouter
+}
+
+func (m *mockProvisionableModule) CaddyModule() ModuleInfo {
+	return ModuleInfo{
+		ID: "admin.api.mock_provision",
+		New: func() Module {
+			mm := &mockProvisionableModule{
+				mockProvisionableRouter: &mockProvisionableRouter{
+					mockRouter:   m.mockRouter,
+					provisionErr: m.provisionErr,
+				},
+			}
+			return mm
+		},
+	}
+}
+
+func TestAdminRouterProvisioning(t *testing.T) {
+	tests := []struct {
+		name         string
+		provisionErr error
+		wantErr      bool
+		routersAfter int // expected number of routers after provisioning
+	}{
+		{
+			name:         "successful provisioning",
+			provisionErr: nil,
+			wantErr:      false,
+			routersAfter: 0,
+		},
+		{
+			name:         "provisioning error",
+			provisionErr: fmt.Errorf("provision failed"),
+			wantErr:      true,
+			routersAfter: 1,
+		},
+	}
+
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			originalModules := make(map[string]ModuleInfo)
+			maps.Copy(originalModules, modules)
+			defer func() {
+				modules = originalModules
+			}()
+
+			mockRoute := AdminRoute{
+				Pattern: "/mock",
+				Handler: AdminHandlerFunc(func(w http.ResponseWriter, r *http.Request) error {
+					return nil
+				}),
+			}
+
+			// Create provisionable module
+			mock := &mockProvisionableModule{
+				mockProvisionableRouter: &mockProvisionableRouter{
+					mockRouter: mockRouter{
+						routes: []AdminRoute{mockRoute},
+					},
+					provisionErr: test.provisionErr,
+				},
+			}
+			RegisterModule(mock)
+
+			admin := &AdminConfig{}
+			addr, err := ParseNetworkAddress("localhost:2019")
+			if err != nil {
+				t.Fatalf("Failed to parse address: %v", err)
+			}
+
+			_ = admin.newAdminHandler(addr, false, Context{})
+			err = admin.provisionAdminRouters(Context{})
+
+			if test.wantErr {
+				if err == nil {
+					t.Error("Expected error but got nil")
+				}
+			} else {
+				if err != nil {
+					t.Errorf("Expected no error but got: %v", err)
+				}
+			}
+
+			if len(admin.routers) != test.routersAfter {
+				t.Errorf("Expected %d routers after provisioning, got %d", test.routersAfter, len(admin.routers))
+			}
+		})
+	}
+}
+
+func TestAllowedOriginsUnixSocket(t *testing.T) {
+	// see comment in allowedOrigins() as to why we do not fill out allowed origins for UDS
+	tests := []struct {
+		name          string
+		addr          NetworkAddress
+		origins       []string
+		expectOrigins []string
+	}{
+		{
+			name: "unix socket with default origins",
+			addr: NetworkAddress{
+				Network: "unix",
+				Host:    "/tmp/caddy.sock",
+			},
+			origins:       nil, // default origins
+			expectOrigins: []string{},
+		},
+		{
+			name: "unix socket with custom origins",
+			addr: NetworkAddress{
+				Network: "unix",
+				Host:    "/tmp/caddy.sock",
+			},
+			origins: []string{"example.com"},
+			expectOrigins: []string{
+				"example.com",
+			},
+		},
+		{
+			name: "tcp socket on localhost gets all loopback addresses",
+			addr: NetworkAddress{
+				Network:   "tcp",
+				Host:      "localhost",
+				StartPort: 2019,
+				EndPort:   2019,
+			},
+			origins: nil,
+			expectOrigins: []string{
+				"localhost:2019",
+				"[::1]:2019",
+				"127.0.0.1:2019",
+			},
+		},
+	}
+
+	for i, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			admin := AdminConfig{
+				Origins: test.origins,
+			}
+
+			got := admin.allowedOrigins(test.addr)
+
+			var gotOrigins []string
+			for _, u := range got {
+				gotOrigins = append(gotOrigins, u.Host)
+			}
+
+			if len(gotOrigins) != len(test.expectOrigins) {
+				t.Errorf("%d: Expected %d origins but got %d", i, len(test.expectOrigins), len(gotOrigins))
+				return
+			}
+
+			expectMap := make(map[string]struct{})
+			for _, origin := range test.expectOrigins {
+				expectMap[origin] = struct{}{}
+			}
+
+			gotMap := make(map[string]struct{})
+			for _, origin := range gotOrigins {
+				gotMap[origin] = struct{}{}
+			}
+
+			if !reflect.DeepEqual(expectMap, gotMap) {
+				t.Errorf("%d: Origins mismatch.\nExpected: %v\nGot: %v", i, test.expectOrigins, gotOrigins)
+			}
+		})
+	}
+}
+
+func TestReplaceRemoteAdminServer(t *testing.T) {
+	const testCert = `MIIDCTCCAfGgAwIBAgIUXsqJ1mY8pKlHQtI3HJ23x2eZPqwwDQYJKoZIhvcNAQEL
+BQAwFDESMBAGA1UEAwwJbG9jYWxob3N0MB4XDTIzMDEwMTAwMDAwMFoXDTI0MDEw
+MTAwMDAwMFowFDESMBAGA1UEAwwJbG9jYWxob3N0MIIBIjANBgkqhkiG9w0BAQEF
+AAOCAQ8AMIIBCgKCAQEA4O4S6BSoYcoxvRqI+h7yPOjF6KjntjzVVm9M+uHK4lzX
+F1L3pSxJ2nDD4wZEV3FJ5yFOHVFqkG2vXG3BIczOlYG7UeNmKbQnKc5kZj3HGUrS
+VGEktA4OJbeZhhWP15gcXN5eDM2eH3g9BFXVX6AURxLiUXzhNBUEZuj/OEyH9yEF
+/qPCE+EjzVvWxvBXwgz/io4r4yok/Vq/bxJ6FlV6R7DX5oJSXyO0VEHZPi9DIyNU
+kK3F/r4U1sWiJGWOs8i3YQWZ2ejh1C0aLFZpPcCGGgMNpoF31gyYP6ZuPDUyCXsE
+g36UUw1JHNtIXYcLhnXuqj4A8TybTDpgXLqvwA9DBQIDAQABo1MwUTAdBgNVHQ4E
+FgQUc13z30pFC63rr/HGKOE7E82vjXwwHwYDVR0jBBgwFoAUc13z30pFC63rr/HG
+KOE7E82vjXwwDwYDVR0TAQH/BAUwAwEB/zANBgkqhkiG9w0BAQsFAAOCAQEAHO3j
+oeiUXXJ7xD4P8Wj5t9d+E8lE1Xv1Dk3Z+EdG5+dan+RcToE42JJp9zB7FIh5Qz8g
+W77LAjqh5oyqz3A2VJcyVgfE3uJP1R1mJM7JfGHf84QH4TZF2Q1RZY4SZs0VQ6+q
+5wSlIZ4NXDy4Q4XkIJBGS61wT8IzYFXYBpx4PCP1Qj0PIE4sevEGwjsBIgxK307o
+BxF8AWe6N6e4YZmQLGjQ+SeH0iwZb6vpkHyAY8Kj2hvK+cq2P7vU3VGi0t3r1F8L
+IvrXHCvO2BMNJ/1UK1M4YNX8LYJqQhg9hEsIROe1OE/m3VhxIYMJI+qZXk9yHfgJ
+vq+SH04xKhtFudVBAQ==`
+
+	tests := []struct {
+		name    string
+		cfg     *Config
+		wantErr bool
+	}{
+		{
+			name:    "nil config",
+			cfg:     nil,
+			wantErr: false,
+		},
+		{
+			name: "nil admin config",
+			cfg: &Config{
+				Admin: nil,
+			},
+			wantErr: false,
+		},
+		{
+			name: "nil remote config",
+			cfg: &Config{
+				Admin: &AdminConfig{},
+			},
+			wantErr: false,
+		},
+		{
+			name: "invalid listen address",
+			cfg: &Config{
+				Admin: &AdminConfig{
+					Remote: &RemoteAdmin{
+						Listen: "invalid:address",
+					},
+				},
+			},
+			wantErr: true,
+		},
+		{
+			name: "valid config",
+			cfg: &Config{
+				Admin: &AdminConfig{
+					Identity: &IdentityConfig{},
+					Remote: &RemoteAdmin{
+						Listen: "localhost:2021",
+						AccessControl: []*AdminAccess{
+							{
+								PublicKeys:  []string{testCert},
+								Permissions: []AdminPermissions{{Methods: []string{"GET"}, Paths: []string{"/test"}}},
+							},
+						},
+					},
+				},
+			},
+			wantErr: false,
+		},
+		{
+			name: "invalid certificate",
+			cfg: &Config{
+				Admin: &AdminConfig{
+					Identity: &IdentityConfig{},
+					Remote: &RemoteAdmin{
+						Listen: "localhost:2021",
+						AccessControl: []*AdminAccess{
+							{
+								PublicKeys:  []string{"invalid-cert-data"},
+								Permissions: []AdminPermissions{{Methods: []string{"GET"}, Paths: []string{"/test"}}},
+							},
+						},
+					},
+				},
+			},
+			wantErr: true,
+		},
+	}
+
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			ctx := Context{
+				Context: context.Background(),
+				cfg:     test.cfg,
+			}
+
+			if test.cfg != nil {
+				test.cfg.storage = &certmagic.FileStorage{Path: t.TempDir()}
+			}
+
+			if test.cfg != nil && test.cfg.Admin != nil && test.cfg.Admin.Identity != nil {
+				identityCertCache = certmagic.NewCache(certmagic.CacheOptions{
+					GetConfigForCert: func(certmagic.Certificate) (*certmagic.Config, error) {
+						return &certmagic.Config{}, nil
+					},
+				})
+			}
+
+			err := replaceRemoteAdminServer(ctx, test.cfg)
+
+			if test.wantErr {
+				if err == nil {
+					t.Error("Expected error but got nil")
+				}
+			} else {
+				if err != nil {
+					t.Errorf("Expected no error but got: %v", err)
+				}
+			}
+
+			// Clean up
+			if remoteAdminServer != nil {
+				_ = stopAdminServer(remoteAdminServer)
+			}
+		})
+	}
+}
+
+type mockIssuer struct {
+	configSet *certmagic.Config
+}
+
+func (m *mockIssuer) Issue(ctx context.Context, csr *x509.CertificateRequest) (*certmagic.IssuedCertificate, error) {
+	return &certmagic.IssuedCertificate{
+		Certificate: []byte(csr.Raw),
+	}, nil
+}
+
+func (m *mockIssuer) SetConfig(cfg *certmagic.Config) {
+	m.configSet = cfg
+}
+
+func (m *mockIssuer) IssuerKey() string {
+	return "mock"
+}
+
+type mockIssuerModule struct {
+	*mockIssuer
+}
+
+func (m *mockIssuerModule) CaddyModule() ModuleInfo {
+	return ModuleInfo{
+		ID: "tls.issuance.acme",
+		New: func() Module {
+			return &mockIssuerModule{mockIssuer: new(mockIssuer)}
+		},
+	}
+}
+
+func TestManageIdentity(t *testing.T) {
+	originalModules := make(map[string]ModuleInfo)
+	maps.Copy(originalModules, modules)
+	defer func() {
+		modules = originalModules
+	}()
+
+	RegisterModule(&mockIssuerModule{})
+
+	certPEM := []byte(`-----BEGIN CERTIFICATE-----
+MIIDujCCAqKgAwIBAgIIE31FZVaPXTUwDQYJKoZIhvcNAQEFBQAwSTELMAkGA1UE
+BhMCVVMxEzARBgNVBAoTCkdvb2dsZSBJbmMxJTAjBgNVBAMTHEdvb2dsZSBJbnRl
+cm5ldCBBdXRob3JpdHkgRzIwHhcNMTQwMTI5MTMyNzQzWhcNMTQwNTI5MDAwMDAw
+WjBpMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2FsaWZvcm5pYTEWMBQGA1UEBwwN
+TW91bnRhaW4gVmlldzETMBEGA1UECgwKR29vZ2xlIEluYzEYMBYGA1UEAwwPbWFp
+bC5nb29nbGUuY29tMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE3lcub2pUwkjC
+5GJQA2ZZfJJi6d1QHhEmkX9VxKYGp6gagZuRqJWy9TXP6++1ZzQQxqZLD0TkuxZ9
+8i9Nz00000CCBjCCAQQwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMGgG
+CCsGAQUFBwEBBFwwWjArBggrBgEFBQcwAoYfaHR0cDovL3BraS5nb29nbGUuY29t
+L0dJQUcyLmNydDArBggrBgEFBQcwAYYfaHR0cDovL2NsaWVudHMxLmdvb2dsZS5j
+b20vb2NzcDAdBgNVHQ4EFgQUiJxtimAuTfwb+aUtBn5UYKreKvMwDAYDVR0TAQH/
+BAIwADAfBgNVHSMEGDAWgBRK3QYWG7z2aLV29YG2u2IaulqBLzAXBgNVHREEEDAO
+ggxtYWlsLmdvb2dsZTANBgkqhkiG9w0BAQUFAAOCAQEAMP6IWgNGZE8wP9TjFjSZ
+3mmW3A1eIr0CuPwNZ2LJ5ZD1i70ojzcj4I9IdP5yPg9CAEV4hNASbM1LzfC7GmJE
+tPzW5tRmpKVWZGRgTgZI8Hp/xZXMwLh9ZmXV4kESFAGj5G5FNvJyUV7R5Eh+7OZX
+7G4jJ4ZGJh+5jzN9HdJJHQHGYNIYOzC7+HH9UMwCjX9vhQ4RjwFZJThS2Yb+y7pb
+9yxTJZoXC6J0H5JpnZb7kZEJ+Xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
+-----END CERTIFICATE-----`)
+
+	keyPEM := []byte(`-----BEGIN PRIVATE KEY-----
+MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDRS0LmTwUT0iwP
+...
+-----END PRIVATE KEY-----`)
+
+	testStorage := certmagic.FileStorage{Path: t.TempDir()}
+	err := testStorage.Store(context.Background(), "localhost/localhost.crt", certPEM)
+	if err != nil {
+		t.Fatal(err)
+	}
+	err = testStorage.Store(context.Background(), "localhost/localhost.key", keyPEM)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	tests := []struct {
+		name       string
+		cfg        *Config
+		wantErr    bool
+		checkState func(*testing.T, *Config)
+	}{
+		{
+			name: "nil config",
+			cfg:  nil,
+		},
+		{
+			name: "nil admin config",
+			cfg: &Config{
+				Admin: nil,
+			},
+		},
+		{
+			name: "nil identity config",
+			cfg: &Config{
+				Admin: &AdminConfig{},
+			},
+		},
+		{
+			name: "default issuer when none specified",
+			cfg: &Config{
+				Admin: &AdminConfig{
+					Identity: &IdentityConfig{
+						Identifiers: []string{"localhost"},
+					},
+				},
+				storage: &testStorage,
+			},
+			checkState: func(t *testing.T, cfg *Config) {
+				if len(cfg.Admin.Identity.issuers) == 0 {
+					t.Error("Expected at least 1 issuer to be configured")
+					return
+				}
+				if _, ok := cfg.Admin.Identity.issuers[0].(*mockIssuerModule); !ok {
+					t.Error("Expected mock issuer to be configured")
+				}
+			},
+		},
+		{
+			name: "custom issuer",
+			cfg: &Config{
+				Admin: &AdminConfig{
+					Identity: &IdentityConfig{
+						Identifiers: []string{"localhost"},
+						IssuersRaw: []json.RawMessage{
+							json.RawMessage(`{"module": "acme"}`),
+						},
+					},
+				},
+				storage: &certmagic.FileStorage{Path: "testdata"},
+			},
+			checkState: func(t *testing.T, cfg *Config) {
+				if len(cfg.Admin.Identity.issuers) != 1 {
+					t.Fatalf("Expected 1 issuer, got %d", len(cfg.Admin.Identity.issuers))
+				}
+				mockIss, ok := cfg.Admin.Identity.issuers[0].(*mockIssuerModule)
+				if !ok {
+					t.Fatal("Expected mock issuer")
+				}
+				if mockIss.configSet == nil {
+					t.Error("Issuer config was not set")
+				}
+			},
+		},
+		{
+			name: "invalid issuer module",
+			cfg: &Config{
+				Admin: &AdminConfig{
+					Identity: &IdentityConfig{
+						Identifiers: []string{"localhost"},
+						IssuersRaw: []json.RawMessage{
+							json.RawMessage(`{"module": "doesnt_exist"}`),
+						},
+					},
+				},
+			},
+			wantErr: true,
+		},
+	}
+
+	for _, test := range tests {
+		t.Run(test.name, func(t *testing.T) {
+			if identityCertCache != nil {
+				// Reset the cert cache before each test
+				identityCertCache.Stop()
+				identityCertCache = nil
+			}
+
+			ctx := Context{
+				Context:         context.Background(),
+				cfg:             test.cfg,
+				moduleInstances: make(map[string][]Module),
+			}
+
+			err := manageIdentity(ctx, test.cfg)
+
+			if test.wantErr {
+				if err == nil {
+					t.Error("Expected error but got nil")
+				}
+				return
+			}
+			if err != nil {
+				t.Fatalf("Expected no error but got: %v", err)
+			}
+
+			if test.checkState != nil {
+				test.checkState(t, test.cfg)
+			}
+		})
+	}
+}

caddy.go

@@ -22,6 +22,7 @@ import (
 	"errors"
 	"fmt"
 	"io"
+	"io/fs"
 	"log"
 	"net/http"
 	"os"
@@ -34,10 +35,12 @@ import (
 	"sync/atomic"
 	"time"
 
-	"github.com/caddyserver/caddy/v2/notify"
 	"github.com/caddyserver/certmagic"
 	"github.com/google/uuid"
 	"go.uber.org/zap"
+
+	"github.com/caddyserver/caddy/v2/internal/filesystems"
+	"github.com/caddyserver/caddy/v2/notify"
 )
 
 // Config is the top (or beginning) of the Caddy configuration structure.
@@ -78,10 +81,17 @@ type Config struct {
 	// associated value.
 	AppsRaw ModuleMap `json:"apps,omitempty" caddy:"namespace="`
 
-	apps    map[string]App
-	storage certmagic.Storage
+	apps map[string]App
+
+	// failedApps is a map of apps that failed to provision with their underlying error.
+	failedApps   map[string]error
+	storage      certmagic.Storage
+	eventEmitter eventEmitter
 
 	cancelFunc context.CancelFunc
+
+	// fileSystems is a dict of fileSystems that will later be loaded from and added to.
+	fileSystems FileSystems
 }
 
 // App is a thing that Caddy runs.
@@ -156,8 +166,8 @@ func changeConfig(method, path string, input []byte, ifMatchHeader string, force
 		return fmt.Errorf("method not allowed")
 	}
 
-	currentCtxMu.Lock()
-	defer currentCtxMu.Unlock()
+	rawCfgMu.Lock()
+	defer rawCfgMu.Unlock()
 
 	if ifMatchHeader != "" {
 		// expect the first and last character to be quotes
@@ -257,8 +267,8 @@ func changeConfig(method, path string, input []byte, ifMatchHeader string, force
 // readConfig traverses the current config to path
 // and writes its JSON encoding to out.
 func readConfig(path string, out io.Writer) error {
-	currentCtxMu.RLock()
-	defer currentCtxMu.RUnlock()
+	rawCfgMu.RLock()
+	defer rawCfgMu.RUnlock()
 	return unsyncedConfigAccess(http.MethodGet, path, nil, out)
 }
 
@@ -305,7 +315,7 @@ func indexConfigObjects(ptr any, configPath string, index map[string]string) err
 // it as the new config, replacing any other current config.
 // It does NOT update the raw config state, as this is a
 // lower-level function; most callers will want to use Load
-// instead. A write lock on currentCtxMu is required! If
+// instead. A write lock on rawCfgMu is required! If
 // allowPersist is false, it will not be persisted to disk,
 // even if it is configured to.
 func unsyncedDecodeAndRun(cfgJSON []byte, allowPersist bool) error {
@@ -314,7 +324,7 @@ func unsyncedDecodeAndRun(cfgJSON []byte, allowPersist bool) error {
 	strippedCfgJSON := RemoveMetaFields(cfgJSON)
 
 	var newCfg *Config
-	err := strictUnmarshalJSON(strippedCfgJSON, &newCfg)
+	err := StrictUnmarshalJSON(strippedCfgJSON, &newCfg)
 	if err != nil {
 		return err
 	}
@@ -340,8 +350,10 @@ func unsyncedDecodeAndRun(cfgJSON []byte, allowPersist bool) error {
 	}
 
 	// swap old context (including its config) with the new one
+	currentCtxMu.Lock()
 	oldCtx := currentCtx
 	currentCtx = ctx
+	currentCtxMu.Unlock()
 
 	// Stop, Cleanup each old app
 	unsyncedStop(oldCtx)
@@ -354,13 +366,13 @@ func unsyncedDecodeAndRun(cfgJSON []byte, allowPersist bool) error {
 			newCfg.Admin.Config.Persist == nil ||
 			*newCfg.Admin.Config.Persist) {
 		dir := filepath.Dir(ConfigAutosavePath)
-		err := os.MkdirAll(dir, 0700)
+		err := os.MkdirAll(dir, 0o700)
 		if err != nil {
 			Log().Error("unable to create folder for config autosave",
 				zap.String("dir", dir),
 				zap.Error(err))
 		} else {
-			err := os.WriteFile(ConfigAutosavePath, cfgJSON, 0600)
+			err := os.WriteFile(ConfigAutosavePath, cfgJSON, 0o600)
 			if err == nil {
 				Log().Info("autosaved config (load with --resume flag)", zap.String("file", ConfigAutosavePath))
 			} else {
@@ -389,6 +401,78 @@ func unsyncedDecodeAndRun(cfgJSON []byte, allowPersist bool) error {
 // will want to use Run instead, which also
 // updates the config's raw state.
 func run(newCfg *Config, start bool) (Context, error) {
+	ctx, err := provisionContext(newCfg, start)
+	if err != nil {
+		globalMetrics.configSuccess.Set(0)
+		return ctx, err
+	}
+
+	if !start {
+		return ctx, nil
+	}
+
+	defer func() {
+		// if newCfg fails to start completely, clean up the already provisioned modules
+		// partially copied from provisionContext
+		if err != nil {
+			globalMetrics.configSuccess.Set(0)
+			ctx.cfg.cancelFunc()
+
+			if currentCtx.cfg != nil {
+				certmagic.Default.Storage = currentCtx.cfg.storage
+			}
+		}
+	}()
+
+	// Provision any admin routers which may need to access
+	// some of the other apps at runtime
+	err = ctx.cfg.Admin.provisionAdminRouters(ctx)
+	if err != nil {
+		return ctx, err
+	}
+
+	// Start
+	err = func() error {
+		started := make([]string, 0, len(ctx.cfg.apps))
+		for name, a := range ctx.cfg.apps {
+			err := a.Start()
+			if err != nil {
+				// an app failed to start, so we need to stop
+				// all other apps that were already started
+				for _, otherAppName := range started {
+					err2 := ctx.cfg.apps[otherAppName].Stop()
+					if err2 != nil {
+						err = fmt.Errorf("%v; additionally, aborting app %s: %v",
+							err, otherAppName, err2)
+					}
+				}
+				return fmt.Errorf("%s app module: start: %v", name, err)
+			}
+			started = append(started, name)
+		}
+		return nil
+	}()
+	if err != nil {
+		return ctx, err
+	}
+	globalMetrics.configSuccess.Set(1)
+	globalMetrics.configSuccessTime.SetToCurrentTime()
+
+	// TODO: This event is experimental and subject to change.
+	ctx.emitEvent("started", nil)
+
+	// now that the user's config is running, finish setting up anything else,
+	// such as remote admin endpoint, config loader, etc.
+	err = finishSettingUp(ctx, ctx.cfg)
+	return ctx, err
+}
+
+// provisionContext creates a new context from the given configuration and provisions
+// storage and apps.
+// If `newCfg` is nil a new empty configuration will be created.
+// If `replaceAdminServer` is true any currently active admin server will be replaced
+// with a new admin server based on the provided configuration.
+func provisionContext(newCfg *Config, replaceAdminServer bool) (Context, error) {
 	// because we will need to roll back any state
 	// modifications if this function errors, we
 	// keep a single error value and scope all
@@ -411,6 +495,7 @@ func run(newCfg *Config, start bool) (Context, error) {
 	ctx, cancel := NewContext(Context{Context: context.Background(), cfg: newCfg})
 	defer func() {
 		if err != nil {
+			globalMetrics.configSuccess.Set(0)
 			// if there were any errors during startup,
 			// we should cancel the new context we created
 			// since the associated config won't be used;
@@ -435,16 +520,12 @@ func run(newCfg *Config, start bool) (Context, error) {
 		return ctx, err
 	}
 
-	// start the admin endpoint (and stop any prior one)
-	if start {
-		err = replaceLocalAdminServer(newCfg)
-		if err != nil {
-			return ctx, fmt.Errorf("starting caddy administration endpoint: %v", err)
-		}
-	}
+	// create the new filesystem map
+	newCfg.fileSystems = &filesystems.FileSystemMap{}
 
 	// prepare the new config for use
 	newCfg.apps = make(map[string]App)
+	newCfg.failedApps = make(map[string]error)
 
 	// set up global storage and make it CertMagic's default storage, too
 	err = func() error {
@@ -471,6 +552,14 @@ func run(newCfg *Config, start bool) (Context, error) {
 		return ctx, err
 	}
 
+	// start the admin endpoint (and stop any prior one)
+	if replaceAdminServer {
+		err = replaceLocalAdminServer(newCfg, ctx)
+		if err != nil {
+			return ctx, fmt.Errorf("starting caddy administration endpoint: %v", err)
+		}
+	}
+
 	// Load and Provision each app and their submodules
 	err = func() error {
 		for appName := range newCfg.AppsRaw {
@@ -480,49 +569,16 @@ func run(newCfg *Config, start bool) (Context, error) {
 		}
 		return nil
 	}()
-	if err != nil {
-		return ctx, err
-	}
+	return ctx, err
+}
 
-	if !start {
-		return ctx, nil
-	}
-
-	// Provision any admin routers which may need to access
-	// some of the other apps at runtime
-	err = newCfg.Admin.provisionAdminRouters(ctx)
-	if err != nil {
-		return ctx, err
-	}
-
-	// Start
-	err = func() error {
-		started := make([]string, 0, len(newCfg.apps))
-		for name, a := range newCfg.apps {
-			err := a.Start()
-			if err != nil {
-				// an app failed to start, so we need to stop
-				// all other apps that were already started
-				for _, otherAppName := range started {
-					err2 := newCfg.apps[otherAppName].Stop()
-					if err2 != nil {
-						err = fmt.Errorf("%v; additionally, aborting app %s: %v",
-							err, otherAppName, err2)
-					}
-				}
-				return fmt.Errorf("%s app module: start: %v", name, err)
-			}
-			started = append(started, name)
-		}
-		return nil
-	}()
-	if err != nil {
-		return ctx, err
-	}
-
-	// now that the user's config is running, finish setting up anything else,
-	// such as remote admin endpoint, config loader, etc.
-	return ctx, finishSettingUp(ctx, newCfg)
+// ProvisionContext creates a new context from the configuration and provisions storage
+// and app modules.
+// The function is intended for testing and advanced use cases only, typically `Run` should be
+// use to ensure a fully functional caddy instance.
+// EXPERIMENTAL: While this is public the interface and implementation details of this function may change.
+func ProvisionContext(newCfg *Config) (Context, error) {
+	return provisionContext(newCfg, false)
 }
 
 // finishSettingUp should be run after all apps have successfully started.
@@ -627,27 +683,43 @@ type ConfigLoader interface {
 // stop the others. Stop should only be called
 // if not replacing with a new config.
 func Stop() error {
+	currentCtxMu.RLock()
+	ctx := currentCtx
+	currentCtxMu.RUnlock()
+
+	rawCfgMu.Lock()
+	unsyncedStop(ctx)
+
 	currentCtxMu.Lock()
-	defer currentCtxMu.Unlock()
-	unsyncedStop(currentCtx)
 	currentCtx = Context{}
+	currentCtxMu.Unlock()
+
 	rawCfgJSON = nil
 	rawCfgIndex = nil
 	rawCfg[rawConfigKey] = nil
+	rawCfgMu.Unlock()
+
 	return nil
 }
 
-// unsyncedStop stops cfg from running, but has
-// no locking around cfg. It is a no-op if cfg is
-// nil. If any app returns an error when stopping,
+// unsyncedStop stops ctx from running, but has
+// no locking around ctx. It is a no-op if ctx has a
+// nil cfg. If any app returns an error when stopping,
 // it is logged and the function continues stopping
 // the next app. This function assumes all apps in
-// cfg were successfully started first.
+// ctx were successfully started first.
+//
+// A lock on rawCfgMu is required, even though this
+// function does not access rawCfg, that lock
+// synchronizes the stop/start of apps.
 func unsyncedStop(ctx Context) {
 	if ctx.cfg == nil {
 		return
 	}
 
+	// TODO: This event is experimental and subject to change.
+	ctx.emitEvent("stopping", nil)
+
 	// stop each app
 	for name, a := range ctx.cfg.apps {
 		err := a.Stop()
@@ -677,8 +749,10 @@ func Validate(cfg *Config) error {
 // Errors are logged along the way, and an appropriate exit
 // code is emitted.
 func exitProcess(ctx context.Context, logger *zap.Logger) {
-	// let the rest of the program know we're quitting
-	atomic.StoreInt32(exiting, 1)
+	// let the rest of the program know we're quitting; only do it once
+	if !atomic.CompareAndSwapInt32(exiting, 0, 1) {
+		return
+	}
 
 	// give the OS or service/process manager our 2 weeks' notice: we quit
 	if err := notify.Stopping(); err != nil {
@@ -691,6 +765,7 @@ func exitProcess(ctx context.Context, logger *zap.Logger) {
 	logger.Warn("exiting; byeee!! 👋")
 
 	exitCode := ExitCodeSuccess
+	lastContext := ActiveContext()
 
 	// stop all apps
 	if err := Stop(); err != nil {
@@ -712,6 +787,16 @@ func exitProcess(ctx context.Context, logger *zap.Logger) {
 		}
 	}
 
+	// execute any process-exit callbacks
+	for _, exitFunc := range lastContext.exitFuncs {
+		exitFunc(ctx)
+	}
+	exitFuncsMu.Lock()
+	for _, exitFunc := range exitFuncs {
+		exitFunc(ctx)
+	}
+	exitFuncsMu.Unlock()
+
 	// shut down admin endpoint(s) in goroutines so that
 	// if this function was called from an admin handler,
 	// it has a chance to return gracefully
@@ -750,6 +835,23 @@ var exiting = new(int32) // accessed atomically
 // EXPERIMENTAL API: subject to change or removal.
 func Exiting() bool { return atomic.LoadInt32(exiting) == 1 }
 
+// OnExit registers a callback to invoke during process exit.
+// This registration is PROCESS-GLOBAL, meaning that each
+// function should only be registered once forever, NOT once
+// per config load (etc).
+//
+// EXPERIMENTAL API: subject to change or removal.
+func OnExit(f func(context.Context)) {
+	exitFuncsMu.Lock()
+	exitFuncs = append(exitFuncs, f)
+	exitFuncsMu.Unlock()
+}
+
+var (
+	exitFuncs   []func(context.Context)
+	exitFuncsMu sync.Mutex
+)
+
 // Duration can be an integer or a string. An integer is
 // interpreted as nanoseconds. If a string, it is a Go
 // time.Duration value such as `300ms`, `1.5h`, or `2h45m`;
@@ -809,14 +911,19 @@ func ParseDuration(s string) (time.Duration, error) {
 // regardless of storage configuration, since each instance is intended to
 // have its own unique ID.
 func InstanceID() (uuid.UUID, error) {
-	uuidFilePath := filepath.Join(AppDataDir(), "instance.uuid")
+	appDataDir := AppDataDir()
+	uuidFilePath := filepath.Join(appDataDir, "instance.uuid")
 	uuidFileBytes, err := os.ReadFile(uuidFilePath)
-	if os.IsNotExist(err) {
+	if errors.Is(err, fs.ErrNotExist) {
 		uuid, err := uuid.NewRandom()
 		if err != nil {
 			return uuid, err
 		}
-		err = os.WriteFile(uuidFilePath, []byte(uuid.String()), 0600)
+		err = os.MkdirAll(appDataDir, 0o700)
+		if err != nil {
+			return uuid, err
+		}
+		err = os.WriteFile(uuidFilePath, []byte(uuid.String()), 0o600)
 		return uuid, err
 	} else if err != nil {
 		return [16]byte{}, err
@@ -824,6 +931,20 @@ func InstanceID() (uuid.UUID, error) {
 	return uuid.ParseBytes(uuidFileBytes)
 }
 
+// CustomVersion is an optional string that overrides Caddy's
+// reported version. It can be helpful when downstream packagers
+// need to manually set Caddy's version. If no other version
+// information is available, the short form version (see
+// Version()) will be set to CustomVersion, and the full version
+// will include CustomVersion at the beginning.
+//
+// Set this variable during `go build` with `-ldflags`:
+//
+//	-ldflags '-X github.com/caddyserver/caddy/v2.CustomVersion=v2.6.2'
+//
+// for example.
+var CustomVersion string
+
 // Version returns the Caddy version in a simple/short form, and
 // a full version string. The short form will not have spaces and
 // is intended for User-Agent strings and similar, but may be
@@ -833,8 +954,10 @@ func InstanceID() (uuid.UUID, error) {
 // build info provided by go.mod dependencies; then it tries to
 // get info from embedded VCS information, which requires having
 // built Caddy from a git repository. If no version is available,
-// this function returns "(devel)" becaise Go uses that, but for
-// the simple form we change it to "unknown".
+// this function returns "(devel)" because Go uses that, but for
+// the simple form we change it to "unknown". If still no version
+// is available (e.g. no VCS repo), then it will use CustomVersion;
+// CustomVersion is always prepended to the full version string.
 //
 // See relevant Go issues: https://github.com/golang/go/issues/29228
 // and https://github.com/golang/go/issues/50603.
@@ -848,13 +971,21 @@ func Version() (simple, full string) {
 	// bi.Main... hopefully.
 	var module *debug.Module
 	bi, ok := debug.ReadBuildInfo()
-	if ok {
-		// find the Caddy module in the dependency list
-		for _, dep := range bi.Deps {
-			if dep.Path == ImportPath {
-				module = dep
-				break
-			}
+	if !ok {
+		if CustomVersion != "" {
+			full = CustomVersion
+			simple = CustomVersion
+			return
+		}
+		full = "unknown"
+		simple = "unknown"
+		return
+	}
+	// find the Caddy module in the dependency list
+	for _, dep := range bi.Deps {
+		if dep.Path == ImportPath {
+			module = dep
+			break
 		}
 	}
 	if module != nil {
@@ -910,13 +1041,119 @@ func Version() (simple, full string) {
 		}
 	}
 
+	if full == "" {
+		if CustomVersion != "" {
+			full = CustomVersion
+		} else {
+			full = "unknown"
+		}
+	} else if CustomVersion != "" {
+		full = CustomVersion + " " + full
+	}
+
 	if simple == "" || simple == "(devel)" {
-		simple = "unknown"
+		if CustomVersion != "" {
+			simple = CustomVersion
+		} else {
+			simple = "unknown"
+		}
 	}
 
 	return
 }
 
+// Event represents something that has happened or is happening.
+// An Event value is not synchronized, so it should be copied if
+// being used in goroutines.
+//
+// EXPERIMENTAL: Events are subject to change.
+type Event struct {
+	// If non-nil, the event has been aborted, meaning
+	// propagation has stopped to other handlers and
+	// the code should stop what it was doing. Emitters
+	// may choose to use this as a signal to adjust their
+	// code path appropriately.
+	Aborted error
+
+	// The data associated with the event. Usually the
+	// original emitter will be the only one to set or
+	// change these values, but the field is exported
+	// so handlers can have full access if needed.
+	// However, this map is not synchronized, so
+	// handlers must not use this map directly in new
+	// goroutines; instead, copy the map to use it in a
+	// goroutine. Data may be nil.
+	Data map[string]any
+
+	id     uuid.UUID
+	ts     time.Time
+	name   string
+	origin Module
+}
+
+// NewEvent creates a new event, but does not emit the event. To emit an
+// event, call Emit() on the current instance of the caddyevents app insteaad.
+//
+// EXPERIMENTAL: Subject to change.
+func NewEvent(ctx Context, name string, data map[string]any) (Event, error) {
+	id, err := uuid.NewRandom()
+	if err != nil {
+		return Event{}, fmt.Errorf("generating new event ID: %v", err)
+	}
+	name = strings.ToLower(name)
+	return Event{
+		Data:   data,
+		id:     id,
+		ts:     time.Now(),
+		name:   name,
+		origin: ctx.Module(),
+	}, nil
+}
+
+func (e Event) ID() uuid.UUID        { return e.id }
+func (e Event) Timestamp() time.Time { return e.ts }
+func (e Event) Name() string         { return e.name }
+func (e Event) Origin() Module       { return e.origin } // Returns the module that originated the event. May be nil, usually if caddy core emits the event.
+
+// CloudEvent exports event e as a structure that, when
+// serialized as JSON, is compatible with the
+// CloudEvents spec.
+func (e Event) CloudEvent() CloudEvent {
+	dataJSON, _ := json.Marshal(e.Data)
+	var source string
+	if e.Origin() == nil {
+		source = "caddy"
+	} else {
+		source = string(e.Origin().CaddyModule().ID)
+	}
+	return CloudEvent{
+		ID:              e.id.String(),
+		Source:          source,
+		SpecVersion:     "1.0",
+		Type:            e.name,
+		Time:            e.ts,
+		DataContentType: "application/json",
+		Data:            dataJSON,
+	}
+}
+
+// CloudEvent is a JSON-serializable structure that
+// is compatible with the CloudEvents specification.
+// See https://cloudevents.io.
+// EXPERIMENTAL: Subject to change.
+type CloudEvent struct {
+	ID              string          `json:"id"`
+	Source          string          `json:"source"`
+	SpecVersion     string          `json:"specversion"`
+	Type            string          `json:"type"`
+	Time            time.Time       `json:"time"`
+	DataContentType string          `json:"datacontenttype,omitempty"`
+	Data            json.RawMessage `json:"data,omitempty"`
+}
+
+// ErrEventAborted cancels an event.
+var ErrEventAborted = errors.New("event aborted")
+
 // ActiveContext returns the currently-active context.
 // This function is experimental and might be changed
 // or removed in the future.
@@ -931,14 +1168,12 @@ type CtxKey string
 
 // This group of variables pertains to the current configuration.
 var (
-	// currentCtxMu protects everything in this var block.
-	currentCtxMu sync.RWMutex
-
 	// currentCtx is the root context for the currently-running
 	// configuration, which can be accessed through this value.
 	// If the Config contained in this value is not nil, then
 	// a config is currently active/running.
-	currentCtx Context
+	currentCtx   Context
+	currentCtxMu sync.RWMutex
 
 	// rawCfg is the current, generic-decoded configuration;
 	// we initialize it as a map with one field ("config")
@@ -956,6 +1191,10 @@ var (
 	// rawCfgIndex is the map of user-assigned ID to expanded
 	// path, for converting /id/ paths to /config/ paths.
 	rawCfgIndex map[string]string
+
+	// rawCfgMu protects all the rawCfg fields and also
+	// essentially synchronizes config changes/reloads.
+	rawCfgMu sync.RWMutex
 )
 
 // errSameConfig is returned if the new config is the same

caddy_test.go

@@ -15,6 +15,7 @@
 package caddy
 
 import (
+	"context"
 	"testing"
 	"time"
 )
@@ -72,3 +73,21 @@ func TestParseDuration(t *testing.T) {
 		}
 	}
 }
+
+func TestEvent_CloudEvent_NilOrigin(t *testing.T) {
+	ctx, _ := NewContext(Context{Context: context.Background()}) // module will be nil by default
+	event, err := NewEvent(ctx, "started", nil)
+	if err != nil {
+		t.Fatalf("NewEvent() error = %v", err)
+	}
+
+	// This should not panic
+	ce := event.CloudEvent()
+
+	if ce.Source != "caddy" {
+		t.Errorf("Expected CloudEvent Source to be 'caddy', got '%s'", ce.Source)
+	}
+	if ce.Type != "started" {
+		t.Errorf("Expected CloudEvent Type to be 'started', got '%s'", ce.Type)
+	}
+}

caddyconfig/caddyfile/adapter.go

@@ -52,9 +52,9 @@ func (a Adapter) Adapt(body []byte, options map[string]any) ([]byte, []caddyconf
 		return nil, warnings, err
 	}
 
-	// lint check: see if input was properly formatted; sometimes messy files files parse
+	// lint check: see if input was properly formatted; sometimes messy files parse
 	// successfully but result in logical errors (the Caddyfile is a bad format, I'm sorry)
-	if warning, different := formattingDifference(filename, body); different {
+	if warning, different := FormattingDifference(filename, body); different {
 		warnings = append(warnings, warning)
 	}
 
@@ -63,12 +63,12 @@ func (a Adapter) Adapt(body []byte, options map[string]any) ([]byte, []caddyconf
 	return result, warnings, err
 }
 
-// formattingDifference returns a warning and true if the formatted version
+// FormattingDifference returns a warning and true if the formatted version
 // is any different from the input; empty warning and false otherwise.
 // TODO: also perform this check on imported files
-func formattingDifference(filename string, body []byte) (caddyconfig.Warning, bool) {
+func FormattingDifference(filename string, body []byte) (caddyconfig.Warning, bool) {
 	// replace windows-style newlines to normalize comparison
-	normalizedBody := bytes.Replace(body, []byte("\r\n"), []byte("\n"), -1)
+	normalizedBody := bytes.ReplaceAll(body, []byte("\r\n"), []byte("\n"))
 
 	formatted := Format(normalizedBody)
 	if bytes.Equal(formatted, normalizedBody) {
@@ -88,34 +88,30 @@ func formattingDifference(filename string, body []byte) (caddyconfig.Warning, bo
 	return caddyconfig.Warning{
 		File:    filename,
 		Line:    line,
-		Message: "Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies",
+		Message: "Caddyfile input is not formatted; run 'caddy fmt --overwrite' to fix inconsistencies",
 	}, true
 }
 
-// Unmarshaler is a type that can unmarshal
-// Caddyfile tokens to set itself up for a
-// JSON encoding. The goal of an unmarshaler
-// is not to set itself up for actual use,
-// but to set itself up for being marshaled
-// into JSON. Caddyfile-unmarshaled values
-// will not be used directly; they will be
-// encoded as JSON and then used from that.
-// Implementations must be able to support
-// multiple segments (instances of their
-// directive or batch of tokens); typically
-// this means wrapping all token logic in
-// a loop: `for d.Next() { ... }`.
+// Unmarshaler is a type that can unmarshal Caddyfile tokens to
+// set itself up for a JSON encoding. The goal of an unmarshaler
+// is not to set itself up for actual use, but to set itself up for
+// being marshaled into JSON. Caddyfile-unmarshaled values will not
+// be used directly; they will be encoded as JSON and then used from
+// that. Implementations _may_ be able to support multiple segments
+// (instances of their directive or batch of tokens); typically this
+// means wrapping parsing logic in a loop: `for d.Next() { ... }`.
+// More commonly, only a single segment is supported, so a simple
+// `d.Next()` at the start should be used to consume the module
+// identifier token (directive name, etc).
 type Unmarshaler interface {
 	UnmarshalCaddyfile(d *Dispenser) error
 }
 
 // ServerType is a type that can evaluate a Caddyfile and set up a caddy config.
 type ServerType interface {
-	// Setup takes the server blocks which
-	// contain tokens, as well as options
-	// (e.g. CLI flags) and creates a Caddy
-	// config, along with any warnings or
-	// an error.
+	// Setup takes the server blocks which contain tokens,
+	// as well as options (e.g. CLI flags) and creates a
+	// Caddy config, along with any warnings or an error.
 	Setup([]ServerBlock, map[string]any) (*caddy.Config, []caddyconfig.Warning, error)
 }
 

caddyconfig/caddyfile/dispenser.go

@@ -30,6 +30,10 @@ type Dispenser struct {
 	tokens  []Token
 	cursor  int
 	nesting int
+
+	// A map of arbitrary context data that can be used
+	// to pass through some information to unmarshalers.
+	context map[string]any
 }
 
 // NewDispenser returns a Dispenser filled with the given tokens.
@@ -101,12 +105,12 @@ func (d *Dispenser) nextOnSameLine() bool {
 		d.cursor++
 		return true
 	}
-	if d.cursor >= len(d.tokens) {
+	if d.cursor >= len(d.tokens)-1 {
 		return false
 	}
-	if d.cursor < len(d.tokens)-1 &&
-		d.tokens[d.cursor].File == d.tokens[d.cursor+1].File &&
-		d.tokens[d.cursor].Line+d.numLineBreaks(d.cursor) == d.tokens[d.cursor+1].Line {
+	curr := d.tokens[d.cursor]
+	next := d.tokens[d.cursor+1]
+	if !isNextOnNewLine(curr, next) {
 		d.cursor++
 		return true
 	}
@@ -122,12 +126,12 @@ func (d *Dispenser) NextLine() bool {
 		d.cursor++
 		return true
 	}
-	if d.cursor >= len(d.tokens) {
+	if d.cursor >= len(d.tokens)-1 {
 		return false
 	}
-	if d.cursor < len(d.tokens)-1 &&
-		(d.tokens[d.cursor].File != d.tokens[d.cursor+1].File ||
-			d.tokens[d.cursor].Line+d.numLineBreaks(d.cursor) < d.tokens[d.cursor+1].Line) {
+	curr := d.tokens[d.cursor]
+	next := d.tokens[d.cursor+1]
+	if isNextOnNewLine(curr, next) {
 		d.cursor++
 		return true
 	}
@@ -203,14 +207,17 @@ func (d *Dispenser) Val() string {
 }
 
 // ValRaw gets the raw text of the current token (including quotes).
+// If the token was a heredoc, then the delimiter is not included,
+// because that is not relevant to any unmarshaling logic at this time.
 // If there is no token loaded, it returns empty string.
 func (d *Dispenser) ValRaw() string {
 	if d.cursor < 0 || d.cursor >= len(d.tokens) {
 		return ""
 	}
 	quote := d.tokens[d.cursor].wasQuoted
-	if quote > 0 {
-		return string(quote) + d.tokens[d.cursor].Text + string(quote) // string literal
+	if quote > 0 && quote != '<' {
+		// string literal
+		return string(quote) + d.tokens[d.cursor].Text + string(quote)
 	}
 	return d.tokens[d.cursor].Text
 }
@@ -388,27 +395,27 @@ func (d *Dispenser) Reset() {
 // an argument.
 func (d *Dispenser) ArgErr() error {
 	if d.Val() == "{" {
-		return d.Err("Unexpected token '{', expecting argument")
+		return d.Err("unexpected token '{', expecting argument")
 	}
-	return d.Errf("Wrong argument count or unexpected line ending after '%s'", d.Val())
+	return d.Errf("wrong argument count or unexpected line ending after '%s'", d.Val())
 }
 
 // SyntaxErr creates a generic syntax error which explains what was
 // found and what was expected.
 func (d *Dispenser) SyntaxErr(expected string) error {
-	msg := fmt.Sprintf("%s:%d - Syntax error: Unexpected token '%s', expecting '%s'", d.File(), d.Line(), d.Val(), expected)
+	msg := fmt.Sprintf("syntax error: unexpected token '%s', expecting '%s', at %s:%d import chain: ['%s']", d.Val(), expected, d.File(), d.Line(), strings.Join(d.Token().imports, "','"))
 	return errors.New(msg)
 }
 
 // EOFErr returns an error indicating that the dispenser reached
 // the end of the input when searching for the next token.
 func (d *Dispenser) EOFErr() error {
-	return d.Errf("Unexpected EOF")
+	return d.Errf("unexpected EOF")
 }
 
 // Err generates a custom parse-time error with a message of msg.
 func (d *Dispenser) Err(msg string) error {
-	return d.Errf(msg)
+	return d.WrapErr(errors.New(msg))
 }
 
 // Errf is like Err, but for formatted error messages
@@ -418,7 +425,10 @@ func (d *Dispenser) Errf(format string, args ...any) error {
 
 // WrapErr takes an existing error and adds the Caddyfile file and line number.
 func (d *Dispenser) WrapErr(err error) error {
-	return fmt.Errorf("%s:%d - Error during parsing: %w", d.File(), d.Line(), err)
+	if len(d.Token().imports) > 0 {
+		return fmt.Errorf("%w, at %s:%d import chain ['%s']", err, d.File(), d.Line(), strings.Join(d.Token().imports, "','"))
+	}
+	return fmt.Errorf("%w, at %s:%d", err, d.File(), d.Line())
 }
 
 // Delete deletes the current token and returns the updated slice
@@ -438,14 +448,42 @@ func (d *Dispenser) Delete() []Token {
 	return d.tokens
 }
 
-// numLineBreaks counts how many line breaks are in the token
-// value given by the token index tknIdx. It returns 0 if the
-// token does not exist or there are no line breaks.
-func (d *Dispenser) numLineBreaks(tknIdx int) int {
-	if tknIdx < 0 || tknIdx >= len(d.tokens) {
-		return 0
+// DeleteN is the same as Delete, but can delete many tokens at once.
+// If there aren't N tokens available to delete, none are deleted.
+func (d *Dispenser) DeleteN(amount int) []Token {
+	if amount > 0 && d.cursor >= (amount-1) && d.cursor <= len(d.tokens)-1 {
+		d.tokens = append(d.tokens[:d.cursor-(amount-1)], d.tokens[d.cursor+1:]...)
+		d.cursor -= amount
 	}
-	return strings.Count(d.tokens[tknIdx].Text, "\n")
+	return d.tokens
+}
+
+// SetContext sets a key-value pair in the context map.
+func (d *Dispenser) SetContext(key string, value any) {
+	if d.context == nil {
+		d.context = make(map[string]any)
+	}
+	d.context[key] = value
+}
+
+// GetContext gets the value of a key in the context map.
+func (d *Dispenser) GetContext(key string) any {
+	if d.context == nil {
+		return nil
+	}
+	return d.context[key]
+}
+
+// GetContextString gets the value of a key in the context map
+// as a string, or an empty string if the key does not exist.
+func (d *Dispenser) GetContextString(key string) string {
+	if d.context == nil {
+		return ""
+	}
+	if val, ok := d.context[key].(string); ok {
+		return val
+	}
+	return ""
 }
 
 // isNewLine determines whether the current token is on a different
@@ -461,25 +499,7 @@ func (d *Dispenser) isNewLine() bool {
 
 	prev := d.tokens[d.cursor-1]
 	curr := d.tokens[d.cursor]
-
-	// If the previous token is from a different file,
-	// we can assume it's from a different line
-	if prev.File != curr.File {
-		return true
-	}
-
-	// The previous token may contain line breaks if
-	// it was quoted and spanned multiple lines. e.g:
-	//
-	// dir "foo
-	//   bar
-	//   baz"
-	prevLineBreaks := d.numLineBreaks(d.cursor - 1)
-
-	// If the previous token (incl line breaks) ends
-	// on a line earlier than the current token,
-	// then the current token is on a new line
-	return prev.Line+prevLineBreaks < curr.Line
+	return isNextOnNewLine(prev, curr)
 }
 
 // isNextOnNewLine determines whether the current token is on a different
@@ -495,23 +515,7 @@ func (d *Dispenser) isNextOnNewLine() bool {
 
 	curr := d.tokens[d.cursor]
 	next := d.tokens[d.cursor+1]
-
-	// If the next token is from a different file,
-	// we can assume it's from a different line
-	if curr.File != next.File {
-		return true
-	}
-
-	// The current token may contain line breaks if
-	// it was quoted and spanned multiple lines. e.g:
-	//
-	// dir "foo
-	//   bar
-	//   baz"
-	currLineBreaks := d.numLineBreaks(d.cursor)
-
-	// If the current token (incl line breaks) ends
-	// on a line earlier than the next token,
-	// then the next token is on a new line
-	return curr.Line+currLineBreaks < next.Line
+	return isNextOnNewLine(curr, next)
 }
+
+const MatcherNameCtxKey = "matcher_name"

caddyconfig/caddyfile/dispenser_test.go

@@ -305,7 +305,7 @@ func TestDispenser_ArgErr_Err(t *testing.T) {
 		t.Errorf("Expected error message with custom message in it ('foobar'); got '%v'", err)
 	}
 
-	var ErrBarIsFull = errors.New("bar is full")
+	ErrBarIsFull := errors.New("bar is full")
 	bookingError := d.Errf("unable to reserve: %w", ErrBarIsFull)
 	if !errors.Is(bookingError, ErrBarIsFull) {
 		t.Errorf("Errf(): should be able to unwrap the error chain")

caddyconfig/caddyfile/formatter.go

@@ -17,6 +17,7 @@ package caddyfile
 import (
 	"bytes"
 	"io"
+	"slices"
 	"unicode"
 )
 
@@ -31,6 +32,14 @@ func Format(input []byte) []byte {
 	out := new(bytes.Buffer)
 	rdr := bytes.NewReader(input)
 
+	type heredocState int
+
+	const (
+		heredocClosed  heredocState = 0
+		heredocOpening heredocState = 1
+		heredocOpened  heredocState = 2
+	)
+
 	var (
 		last rune // the last character that was written to the result
 
@@ -47,7 +56,13 @@ func Format(input []byte) []byte {
 		quoted  bool // whether we're in a quoted segment
 		escaped bool // whether current char is escaped
 
-		nesting int // indentation level
+		heredoc              heredocState // whether we're in a heredoc
+		heredocEscaped       bool         // whether heredoc is escaped
+		heredocMarker        []rune
+		heredocClosingMarker []rune
+
+		nesting         int // indentation level
+		withinBackquote bool
 	)
 
 	write := func(ch rune) {
@@ -74,6 +89,65 @@ func Format(input []byte) []byte {
 			}
 			panic(err)
 		}
+		if ch == '`' {
+			withinBackquote = !withinBackquote
+		}
+
+		// detect whether we have the start of a heredoc
+		if !quoted && (heredoc == heredocClosed && !heredocEscaped) &&
+			space && last == '<' && ch == '<' {
+			write(ch)
+			heredoc = heredocOpening
+			space = false
+			continue
+		}
+
+		if heredoc == heredocOpening {
+			if ch == '\n' {
+				if len(heredocMarker) > 0 && heredocMarkerRegexp.MatchString(string(heredocMarker)) {
+					heredoc = heredocOpened
+				} else {
+					heredocMarker = nil
+					heredoc = heredocClosed
+					nextLine()
+					continue
+				}
+				write(ch)
+				continue
+			}
+			if unicode.IsSpace(ch) {
+				// a space means it's just a regular token and not a heredoc
+				heredocMarker = nil
+				heredoc = heredocClosed
+			} else {
+				heredocMarker = append(heredocMarker, ch)
+				write(ch)
+				continue
+			}
+		}
+		// if we're in a heredoc, all characters are read&write as-is
+		if heredoc == heredocOpened {
+			heredocClosingMarker = append(heredocClosingMarker, ch)
+			if len(heredocClosingMarker) > len(heredocMarker)+1 { // We assert that the heredocClosingMarker is followed by a unicode.Space
+				heredocClosingMarker = heredocClosingMarker[1:]
+			}
+			// check if we're done
+			if unicode.IsSpace(ch) && slices.Equal(heredocClosingMarker[:len(heredocClosingMarker)-1], heredocMarker) {
+				heredocMarker = nil
+				heredocClosingMarker = nil
+				heredoc = heredocClosed
+			} else {
+				write(ch)
+				if ch == '\n' {
+					heredocClosingMarker = heredocClosingMarker[:0]
+				}
+				continue
+			}
+		}
+
+		if last == '<' && space {
+			space = false
+		}
 
 		if comment {
 			if ch == '\n' {
@@ -98,6 +172,9 @@ func Format(input []byte) []byte {
 		}
 
 		if escaped {
+			if ch == '<' {
+				heredocEscaped = true
+			}
 			write(ch)
 			escaped = false
 			continue
@@ -117,6 +194,7 @@ func Format(input []byte) []byte {
 
 		if unicode.IsSpace(ch) {
 			space = true
+			heredocEscaped = false
 			if ch == '\n' {
 				newLines++
 			}
@@ -162,14 +240,23 @@ func Format(input []byte) []byte {
 		switch {
 		case ch == '{':
 			openBrace = true
-			openBraceWritten = false
 			openBraceSpace = spacePrior && !beginningOfLine
 			if openBraceSpace {
 				write(' ')
 			}
+			openBraceWritten = false
+			if withinBackquote {
+				write('{')
+				openBraceWritten = true
+				continue
+			}
 			continue
 
 		case ch == '}' && (spacePrior || !openBrace):
+			if withinBackquote {
+				write('}')
+				continue
+			}
 			if last != '\n' {
 				nextLine()
 			}
@@ -205,6 +292,11 @@ func Format(input []byte) []byte {
 			write('{')
 			openBraceWritten = true
 		}
+
+		if spacePrior && ch == '<' {
+			space = true
+		}
+
 		write(ch)
 
 		beginningOfLine = false

caddyconfig/caddyfile/formatter_test.go

@@ -364,6 +364,86 @@ block {
 }
 `,
 		},
+		{
+			description: "keep heredoc as-is",
+			input: `block {
+	heredoc <<HEREDOC
+	Here's more than one space       Here's more than one space
+	HEREDOC
+}
+`,
+			expect: `block {
+	heredoc <<HEREDOC
+	Here's more than one space       Here's more than one space
+	HEREDOC
+}
+`,
+		},
+		{
+			description: "Mixing heredoc with regular part",
+			input: `block {
+	heredoc <<HEREDOC
+	Here's more than one space       Here's more than one space
+	HEREDOC
+	respond "More than one space will be eaten"     200
+}
+
+block2 {
+	heredoc <<HEREDOC
+	Here's more than one space       Here's more than one space
+	HEREDOC
+	respond "More than one space will be eaten" 200
+}
+`,
+			expect: `block {
+	heredoc <<HEREDOC
+	Here's more than one space       Here's more than one space
+	HEREDOC
+	respond "More than one space will be eaten" 200
+}
+
+block2 {
+	heredoc <<HEREDOC
+	Here's more than one space       Here's more than one space
+	HEREDOC
+	respond "More than one space will be eaten" 200
+}
+`,
+		},
+		{
+			description: "Heredoc as regular token",
+			input: `block {
+	heredoc <<HEREDOC                                 "More than one space will be eaten"
+}
+`,
+			expect: `block {
+	heredoc <<HEREDOC "More than one space will be eaten"
+}
+`,
+		},
+		{
+			description: "Escape heredoc",
+			input: `block {
+	heredoc \<<HEREDOC
+	respond "More than one space will be eaten"                           200
+}
+`,
+			expect: `block {
+	heredoc \<<HEREDOC
+	respond "More than one space will be eaten" 200
+}
+`,
+		},
+		{
+			description: "Preserve braces wrapped by backquotes",
+			input:       "block {respond `All braces should remain: {{now | date \"2006\"}}`}",
+			expect:      "block {respond `All braces should remain: {{now | date \"2006\"}}`}",
+		},
+		{
+			description: "Preserve braces wrapped by quotes",
+			input:       "block {respond \"All braces should remain: {{now | date `2006`}}\"}",
+			expect:      "block {respond \"All braces should remain: {{now | date `2006`}}\"}",
+		},
 	} {
 		// the formatter should output a trailing newline,
 		// even if the tests aren't written to expect that

caddyconfig/caddyfile/importargs.go

@@ -0,0 +1,160 @@
+// Copyright 2015 Matthew Holt and The Caddy Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package caddyfile
+
+import (
+	"regexp"
+	"strconv"
+	"strings"
+
+	"go.uber.org/zap"
+
+	"github.com/caddyserver/caddy/v2"
+)
+
+// parseVariadic determines if the token is a variadic placeholder,
+// and if so, determines the index range (start/end) of args to use.
+// Returns a boolean signaling whether a variadic placeholder was found,
+// and the start and end indices.
+func parseVariadic(token Token, argCount int) (bool, int, int) {
+	if !strings.HasPrefix(token.Text, "{args[") {
+		return false, 0, 0
+	}
+	if !strings.HasSuffix(token.Text, "]}") {
+		return false, 0, 0
+	}
+
+	argRange := strings.TrimSuffix(strings.TrimPrefix(token.Text, "{args["), "]}")
+	if argRange == "" {
+		caddy.Log().Named("caddyfile").Warn(
+			"Placeholder "+token.Text+" cannot have an empty index",
+			zap.String("file", token.File+":"+strconv.Itoa(token.Line)), zap.Strings("import_chain", token.imports))
+		return false, 0, 0
+	}
+
+	start, end, found := strings.Cut(argRange, ":")
+
+	// If no ":" delimiter is found, this is not a variadic.
+	// The replacer will pick this up.
+	if !found {
+		return false, 0, 0
+	}
+
+	// A valid token may contain several placeholders, and
+	// they may be separated by ":". It's not variadic.
+	// https://github.com/caddyserver/caddy/issues/5716
+	if strings.Contains(start, "}") || strings.Contains(end, "{") {
+		return false, 0, 0
+	}
+
+	var (
+		startIndex = 0
+		endIndex   = argCount
+		err        error
+	)
+	if start != "" {
+		startIndex, err = strconv.Atoi(start)
+		if err != nil {
+			caddy.Log().Named("caddyfile").Warn(
+				"Variadic placeholder "+token.Text+" has an invalid start index",
+				zap.String("file", token.File+":"+strconv.Itoa(token.Line)), zap.Strings("import_chain", token.imports))
+			return false, 0, 0
+		}
+	}
+	if end != "" {
+		endIndex, err = strconv.Atoi(end)
+		if err != nil {
+			caddy.Log().Named("caddyfile").Warn(
+				"Variadic placeholder "+token.Text+" has an invalid end index",
+				zap.String("file", token.File+":"+strconv.Itoa(token.Line)), zap.Strings("import_chain", token.imports))
+			return false, 0, 0
+		}
+	}
+
+	// bound check
+	if startIndex < 0 || startIndex > endIndex || endIndex > argCount {
+		caddy.Log().Named("caddyfile").Warn(
+			"Variadic placeholder "+token.Text+" indices are out of bounds, only "+strconv.Itoa(argCount)+" argument(s) exist",
+			zap.String("file", token.File+":"+strconv.Itoa(token.Line)), zap.Strings("import_chain", token.imports))
+		return false, 0, 0
+	}
+	return true, startIndex, endIndex
+}
+
+// makeArgsReplacer prepares a Replacer which can replace
+// non-variadic args placeholders in imported tokens.
+func makeArgsReplacer(args []string) *caddy.Replacer {
+	repl := caddy.NewEmptyReplacer()
+	repl.Map(func(key string) (any, bool) {
+		// TODO: Remove the deprecated {args.*} placeholder
+		// support at some point in the future
+		if matches := argsRegexpIndexDeprecated.FindStringSubmatch(key); len(matches) > 0 {
+			// What's matched may be a substring of the key
+			if matches[0] != key {
+				return nil, false
+			}
+
+			value, err := strconv.Atoi(matches[1])
+			if err != nil {
+				caddy.Log().Named("caddyfile").Warn(
+					"Placeholder {args." + matches[1] + "} has an invalid index")
+				return nil, false
+			}
+			if value >= len(args) {
+				caddy.Log().Named("caddyfile").Warn(
+					"Placeholder {args." + matches[1] + "} index is out of bounds, only " + strconv.Itoa(len(args)) + " argument(s) exist")
+				return nil, false
+			}
+			caddy.Log().Named("caddyfile").Warn(
+				"Placeholder {args." + matches[1] + "} deprecated, use {args[" + matches[1] + "]} instead")
+			return args[value], true
+		}
+
+		// Handle args[*] form
+		if matches := argsRegexpIndex.FindStringSubmatch(key); len(matches) > 0 {
+			// What's matched may be a substring of the key
+			if matches[0] != key {
+				return nil, false
+			}
+
+			if strings.Contains(matches[1], ":") {
+				caddy.Log().Named("caddyfile").Warn(
+					"Variadic placeholder {args[" + matches[1] + "]} must be a token on its own")
+				return nil, false
+			}
+			value, err := strconv.Atoi(matches[1])
+			if err != nil {
+				caddy.Log().Named("caddyfile").Warn(
+					"Placeholder {args[" + matches[1] + "]} has an invalid index")
+				return nil, false
+			}
+			if value >= len(args) {
+				caddy.Log().Named("caddyfile").Warn(
+					"Placeholder {args[" + matches[1] + "]} index is out of bounds, only " + strconv.Itoa(len(args)) + " argument(s) exist")
+				return nil, false
+			}
+			return args[value], true
+		}
+
+		// Not an args placeholder, ignore
+		return nil, false
+	})
+	return repl
+}
+
+var (
+	argsRegexpIndexDeprecated = regexp.MustCompile(`args\.(.+)`)
+	argsRegexpIndex           = regexp.MustCompile(`args\[(.+)]`)
+)

caddyconfig/caddyfile/importgraph.go

@@ -16,24 +16,26 @@ package caddyfile
 
 import (
 	"fmt"
+	"slices"
 )
 
 type adjacency map[string][]string
 
 type importGraph struct {
-	nodes map[string]bool
+	nodes map[string]struct{}
 	edges adjacency
 }
 
 func (i *importGraph) addNode(name string) {
 	if i.nodes == nil {
-		i.nodes = make(map[string]bool)
+		i.nodes = make(map[string]struct{})
 	}
 	if _, exists := i.nodes[name]; exists {
 		return
 	}
-	i.nodes[name] = true
+	i.nodes[name] = struct{}{}
 }
+
 func (i *importGraph) addNodes(names []string) {
 	for _, name := range names {
 		i.addNode(name)
@@ -43,6 +45,7 @@ func (i *importGraph) addNodes(names []string) {
 func (i *importGraph) removeNode(name string) {
 	delete(i.nodes, name)
 }
+
 func (i *importGraph) removeNodes(names []string) {
 	for _, name := range names {
 		i.removeNode(name)
@@ -64,7 +67,7 @@ func (i *importGraph) addEdge(from, to string) error {
 	}
 
 	if i.nodes == nil {
-		i.nodes = make(map[string]bool)
+		i.nodes = make(map[string]struct{})
 	}
 	if i.edges == nil {
 		i.edges = make(adjacency)
@@ -73,6 +76,7 @@ func (i *importGraph) addEdge(from, to string) error {
 	i.edges[from] = append(i.edges[from], to)
 	return nil
 }
+
 func (i *importGraph) addEdges(from string, tos []string) error {
 	for _, to := range tos {
 		err := i.addEdge(from, to)
@@ -88,12 +92,7 @@ func (i *importGraph) areConnected(from, to string) bool {
 	if !ok {
 		return false
 	}
-	for _, v := range al {
-		if v == to {
-			return true
-		}
-	}
-	return false
+	return slices.Contains(al, to)
 }
 
 func (i *importGraph) willCycle(from, to string) bool {

caddyconfig/caddyfile/lexer.go

@@ -17,7 +17,10 @@ package caddyfile
 import (
 	"bufio"
 	"bytes"
+	"fmt"
 	"io"
+	"regexp"
+	"strings"
 	"unicode"
 )
 
@@ -35,15 +38,41 @@ type (
 
 	// Token represents a single parsable unit.
 	Token struct {
-		File        string
-		Line        int
-		Text        string
-		wasQuoted   rune // enclosing quote character, if any
-		inSnippet   bool
-		snippetName string
+		File          string
+		imports       []string
+		Line          int
+		Text          string
+		wasQuoted     rune // enclosing quote character, if any
+		heredocMarker string
+		snippetName   string
 	}
 )
 
+// Tokenize takes bytes as input and lexes it into
+// a list of tokens that can be parsed as a Caddyfile.
+// Also takes a filename to fill the token's File as
+// the source of the tokens, which is important to
+// determine relative paths for `import` directives.
+func Tokenize(input []byte, filename string) ([]Token, error) {
+	l := lexer{}
+	if err := l.load(bytes.NewReader(input)); err != nil {
+		return nil, err
+	}
+	var tokens []Token
+	for {
+		found, err := l.next()
+		if err != nil {
+			return nil, err
+		}
+		if !found {
+			break
+		}
+		l.token.File = filename
+		tokens = append(tokens, l.token)
+	}
+	return tokens, nil
+}
+
 // load prepares the lexer to scan an input for tokens.
 // It discards any leading byte order mark.
 func (l *lexer) load(input io.Reader) error {
@@ -75,28 +104,107 @@ func (l *lexer) load(input io.Reader) error {
 // may be escaped. The rest of the line is skipped
 // if a "#" character is read in. Returns true if
 // a token was loaded; false otherwise.
-func (l *lexer) next() bool {
+func (l *lexer) next() (bool, error) {
 	var val []rune
-	var comment, quoted, btQuoted, escaped bool
+	var comment, quoted, btQuoted, inHeredoc, heredocEscaped, escaped bool
+	var heredocMarker string
 
 	makeToken := func(quoted rune) bool {
 		l.token.Text = string(val)
 		l.token.wasQuoted = quoted
+		l.token.heredocMarker = heredocMarker
 		return true
 	}
 
 	for {
+		// Read a character in; if err then if we had
+		// read some characters, make a token. If we
+		// reached EOF, then no more tokens to read.
+		// If no EOF, then we had a problem.
 		ch, _, err := l.reader.ReadRune()
 		if err != nil {
 			if len(val) > 0 {
-				return makeToken(0)
+				if inHeredoc {
+					return false, fmt.Errorf("incomplete heredoc <<%s on line #%d, expected ending marker %s", heredocMarker, l.line+l.skippedLines, heredocMarker)
+				}
+
+				return makeToken(0), nil
 			}
 			if err == io.EOF {
-				return false
+				return false, nil
 			}
-			panic(err)
+			return false, err
 		}
 
+		// detect whether we have the start of a heredoc
+		if (!quoted && !btQuoted) && (!inHeredoc && !heredocEscaped) &&
+			len(val) > 1 && string(val[:2]) == "<<" {
+			// a space means it's just a regular token and not a heredoc
+			if ch == ' ' {
+				return makeToken(0), nil
+			}
+
+			// skip CR, we only care about LF
+			if ch == '\r' {
+				continue
+			}
+
+			// after hitting a newline, we know that the heredoc marker
+			// is the characters after the two << and the newline.
+			// we reset the val because the heredoc is syntax we don't
+			// want to keep.
+			if ch == '\n' {
+				if len(val) == 2 {
+					return false, fmt.Errorf("missing opening heredoc marker on line #%d; must contain only alpha-numeric characters, dashes and underscores; got empty string", l.line)
+				}
+
+				// check if there's too many <
+				if string(val[:3]) == "<<<" {
+					return false, fmt.Errorf("too many '<' for heredoc on line #%d; only use two, for example <<END", l.line)
+				}
+
+				heredocMarker = string(val[2:])
+				if !heredocMarkerRegexp.Match([]byte(heredocMarker)) {
+					return false, fmt.Errorf("heredoc marker on line #%d must contain only alpha-numeric characters, dashes and underscores; got '%s'", l.line, heredocMarker)
+				}
+
+				inHeredoc = true
+				l.skippedLines++
+				val = nil
+				continue
+			}
+			val = append(val, ch)
+			continue
+		}
+
+		// if we're in a heredoc, all characters are read as-is
+		if inHeredoc {
+			val = append(val, ch)
+
+			if ch == '\n' {
+				l.skippedLines++
+			}
+
+			// check if we're done, i.e. that the last few characters are the marker
+			if len(val) >= len(heredocMarker) && heredocMarker == string(val[len(val)-len(heredocMarker):]) {
+				// set the final value
+				val, err = l.finalizeHeredoc(val, heredocMarker)
+				if err != nil {
+					return false, err
+				}
+
+				// set the line counter, and make the token
+				l.line += l.skippedLines
+				l.skippedLines = 0
+				return makeToken('<'), nil
+			}
+
+			// stay in the heredoc until we find the ending marker
+			continue
+		}
+
+		// track whether we found an escape '\' for the next
+		// iteration to be contextually aware
 		if !escaped && !btQuoted && ch == '\\' {
 			escaped = true
 			continue
@@ -111,26 +219,29 @@ func (l *lexer) next() bool {
 				}
 				escaped = false
 			} else {
-				if quoted && ch == '"' {
-					return makeToken('"')
-				}
-				if btQuoted && ch == '`' {
-					return makeToken('`')
+				if (quoted && ch == '"') || (btQuoted && ch == '`') {
+					return makeToken(ch), nil
 				}
 			}
+			// allow quoted text to wrap continue on multiple lines
 			if ch == '\n' {
 				l.line += 1 + l.skippedLines
 				l.skippedLines = 0
 			}
+			// collect this character as part of the quoted token
 			val = append(val, ch)
 			continue
 		}
 
 		if unicode.IsSpace(ch) {
+			// ignore CR altogether, we only actually care about LF (\n)
 			if ch == '\r' {
 				continue
 			}
+			// end of the line
 			if ch == '\n' {
+				// newlines can be escaped to chain arguments
+				// onto multiple lines; else, increment the line count
 				if escaped {
 					l.skippedLines++
 					escaped = false
@@ -138,14 +249,18 @@ func (l *lexer) next() bool {
 					l.line += 1 + l.skippedLines
 					l.skippedLines = 0
 				}
+				// comments (#) are single-line only
 				comment = false
 			}
+			// any kind of space means we're at the end of this token
 			if len(val) > 0 {
-				return makeToken(0)
+				return makeToken(0), nil
 			}
 			continue
 		}
 
+		// comments must be at the start of a token,
+		// in other words, preceded by space or newline
 		if ch == '#' && len(val) == 0 {
 			comment = true
 		}
@@ -166,7 +281,12 @@ func (l *lexer) next() bool {
 		}
 
 		if escaped {
-			val = append(val, '\\')
+			// allow escaping the first < to skip the heredoc syntax
+			if ch == '<' {
+				heredocEscaped = true
+			} else {
+				val = append(val, '\\')
+			}
 			escaped = false
 		}
 
@@ -174,24 +294,107 @@ func (l *lexer) next() bool {
 	}
 }
 
-// Tokenize takes bytes as input and lexes it into
-// a list of tokens that can be parsed as a Caddyfile.
-// Also takes a filename to fill the token's File as
-// the source of the tokens, which is important to
-// determine relative paths for `import` directives.
-func Tokenize(input []byte, filename string) ([]Token, error) {
-	l := lexer{}
-	if err := l.load(bytes.NewReader(input)); err != nil {
-		return nil, err
+// finalizeHeredoc takes the runes read as the heredoc text and the marker,
+// and processes the text to strip leading whitespace, returning the final
+// value without the leading whitespace.
+func (l *lexer) finalizeHeredoc(val []rune, marker string) ([]rune, error) {
+	stringVal := string(val)
+
+	// find the last newline of the heredoc, which is where the contents end
+	lastNewline := strings.LastIndex(stringVal, "\n")
+
+	// collapse the content, then split into separate lines
+	lines := strings.Split(stringVal[:lastNewline+1], "\n")
+
+	// figure out how much whitespace we need to strip from the front of every line
+	// by getting the string that precedes the marker, on the last line
+	paddingToStrip := stringVal[lastNewline+1 : len(stringVal)-len(marker)]
+
+	// iterate over each line and strip the whitespace from the front
+	var out string
+	for lineNum, lineText := range lines[:len(lines)-1] {
+		if lineText == "" || lineText == "\r" {
+			out += "\n"
+			continue
+		}
+
+		// find an exact match for the padding
+		index := strings.Index(lineText, paddingToStrip)
+
+		// if the padding doesn't match exactly at the start then we can't safely strip
+		if index != 0 {
+			cleanLineText := strings.TrimRight(lineText, "\r\n")
+			return nil, fmt.Errorf("mismatched leading whitespace in heredoc <<%s on line #%d [%s], expected whitespace [%s] to match the closing marker", marker, l.line+lineNum+1, cleanLineText, paddingToStrip)
+		}
+
+		// strip, then append the line, with the newline, to the output.
+		// also removes all "\r" because Windows.
+		out += strings.ReplaceAll(lineText[len(paddingToStrip):]+"\n", "\r", "")
 	}
-	var tokens []Token
-	for l.next() {
-		l.token.File = filename
-		tokens = append(tokens, l.token)
+
+	// Remove the trailing newline from the loop
+	if len(out) > 0 && out[len(out)-1] == '\n' {
+		out = out[:len(out)-1]
 	}
-	return tokens, nil
+
+	// return the final value
+	return []rune(out), nil
 }
 
+// Quoted returns true if the token was enclosed in quotes
+// (i.e. double quotes, backticks, or heredoc).
 func (t Token) Quoted() bool {
 	return t.wasQuoted > 0
 }
+
+// NumLineBreaks counts how many line breaks are in the token text.
+func (t Token) NumLineBreaks() int {
+	lineBreaks := strings.Count(t.Text, "\n")
+	if t.wasQuoted == '<' {
+		// heredocs have an extra linebreak because the opening
+		// delimiter is on its own line and is not included in the
+		// token Text itself, and the trailing newline is removed.
+		lineBreaks += 2
+	}
+	return lineBreaks
+}
+
+// Clone returns a deep copy of the token.
+func (t Token) Clone() Token {
+	return Token{
+		File:          t.File,
+		imports:       append([]string{}, t.imports...),
+		Line:          t.Line,
+		Text:          t.Text,
+		wasQuoted:     t.wasQuoted,
+		heredocMarker: t.heredocMarker,
+		snippetName:   t.snippetName,
+	}
+}
+
+var heredocMarkerRegexp = regexp.MustCompile("^[A-Za-z0-9_-]+$")
+
+// isNextOnNewLine tests whether t2 is on a different line from t1
+func isNextOnNewLine(t1, t2 Token) bool {
+	// If the second token is from a different file,
+	// we can assume it's from a different line
+	if t1.File != t2.File {
+		return true
+	}
+
+	// If the second token is from a different import chain,
+	// we can assume it's from a different line
+	if len(t1.imports) != len(t2.imports) {
+		return true
+	}
+	for i, im := range t1.imports {
+		if im != t2.imports[i] {
+			return true
+		}
+	}
+
+	// If the first token (incl line breaks) ends
+	// on a line earlier than the next token,
+	// then the second token is on a new line
+	return t1.Line+t1.NumLineBreaks() < t2.Line
+}

caddyconfig/caddyfile/lexer_test.go

@@ -18,13 +18,13 @@ import (
 	"testing"
 )
 
-type lexerTestCase struct {
-	input    []byte
-	expected []Token
-}
-
 func TestLexer(t *testing.T) {
-	testCases := []lexerTestCase{
+	testCases := []struct {
+		input        []byte
+		expected     []Token
+		expectErr    bool
+		errorMessage string
+	}{
 		{
 			input: []byte(`host:123`),
 			expected: []Token{
@@ -249,12 +249,273 @@ func TestLexer(t *testing.T) {
 				{Line: 1, Text: `quotes`},
 			},
 		},
+		{
+			input: []byte(`heredoc <<EOF
+content
+EOF same-line-arg
+	`),
+			expected: []Token{
+				{Line: 1, Text: `heredoc`},
+				{Line: 1, Text: "content"},
+				{Line: 3, Text: `same-line-arg`},
+			},
+		},
+		{
+			input: []byte(`heredoc <<VERY-LONG-MARKER
+content
+VERY-LONG-MARKER same-line-arg
+	`),
+			expected: []Token{
+				{Line: 1, Text: `heredoc`},
+				{Line: 1, Text: "content"},
+				{Line: 3, Text: `same-line-arg`},
+			},
+		},
+		{
+			input: []byte(`heredoc <<EOF
+extra-newline
+
+EOF same-line-arg
+	`),
+			expected: []Token{
+				{Line: 1, Text: `heredoc`},
+				{Line: 1, Text: "extra-newline\n"},
+				{Line: 4, Text: `same-line-arg`},
+			},
+		},
+		{
+			input: []byte(`heredoc <<EOF
+EOF
+	HERE same-line-arg
+	`),
+			expected: []Token{
+				{Line: 1, Text: `heredoc`},
+				{Line: 1, Text: ``},
+				{Line: 3, Text: `HERE`},
+				{Line: 3, Text: `same-line-arg`},
+			},
+		},
+		{
+			input: []byte(`heredoc <<EOF
+		EOF same-line-arg
+	`),
+			expected: []Token{
+				{Line: 1, Text: `heredoc`},
+				{Line: 1, Text: ""},
+				{Line: 2, Text: `same-line-arg`},
+			},
+		},
+		{
+			input: []byte(`heredoc <<EOF
+	content
+	EOF same-line-arg
+	`),
+			expected: []Token{
+				{Line: 1, Text: `heredoc`},
+				{Line: 1, Text: "content"},
+				{Line: 3, Text: `same-line-arg`},
+			},
+		},
+		{
+			input: []byte(`prev-line
+	heredoc <<EOF
+		multi
+		line
+		content
+	EOF same-line-arg
+	next-line
+	`),
+			expected: []Token{
+				{Line: 1, Text: `prev-line`},
+				{Line: 2, Text: `heredoc`},
+				{Line: 2, Text: "\tmulti\n\tline\n\tcontent"},
+				{Line: 6, Text: `same-line-arg`},
+				{Line: 7, Text: `next-line`},
+			},
+		},
+		{
+			input: []byte(`escaped-heredoc \<< >>`),
+			expected: []Token{
+				{Line: 1, Text: `escaped-heredoc`},
+				{Line: 1, Text: `<<`},
+				{Line: 1, Text: `>>`},
+			},
+		},
+		{
+			input: []byte(`not-a-heredoc <EOF
+	content
+	`),
+			expected: []Token{
+				{Line: 1, Text: `not-a-heredoc`},
+				{Line: 1, Text: `<EOF`},
+				{Line: 2, Text: `content`},
+			},
+		},
+		{
+			input: []byte(`not-a-heredoc <<<EOF content`),
+			expected: []Token{
+				{Line: 1, Text: `not-a-heredoc`},
+				{Line: 1, Text: `<<<EOF`},
+				{Line: 1, Text: `content`},
+			},
+		},
+		{
+			input: []byte(`not-a-heredoc "<<" ">>"`),
+			expected: []Token{
+				{Line: 1, Text: `not-a-heredoc`},
+				{Line: 1, Text: `<<`},
+				{Line: 1, Text: `>>`},
+			},
+		},
+		{
+			input: []byte(`not-a-heredoc << >>`),
+			expected: []Token{
+				{Line: 1, Text: `not-a-heredoc`},
+				{Line: 1, Text: `<<`},
+				{Line: 1, Text: `>>`},
+			},
+		},
+		{
+			input: []byte(`not-a-heredoc <<HERE SAME LINE
+	content
+	HERE same-line-arg
+	`),
+			expected: []Token{
+				{Line: 1, Text: `not-a-heredoc`},
+				{Line: 1, Text: `<<HERE`},
+				{Line: 1, Text: `SAME`},
+				{Line: 1, Text: `LINE`},
+				{Line: 2, Text: `content`},
+				{Line: 3, Text: `HERE`},
+				{Line: 3, Text: `same-line-arg`},
+			},
+		},
+		{
+			input: []byte(`heredoc <<s
+			<EFBFBD>
+			s
+	`),
+			expected: []Token{
+				{Line: 1, Text: `heredoc`},
+				{Line: 1, Text: "<22>"},
+			},
+		},
+		{
+			input: []byte("\u000Aheredoc \u003C\u003C\u0073\u0073\u000A\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F\u000A\u0073\u0073\u000A\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F\u000A\u00BF\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F"),
+			expected: []Token{
+				{
+					Line: 2,
+					Text: "heredoc",
+				},
+				{
+					Line: 2,
+					Text: "\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F",
+				},
+				{
+					Line: 5,
+					Text: "\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F",
+				},
+				{
+					Line: 6,
+					Text: "\u00BF\u00BF\u0057\u0001\u0000\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u00FF\u003D\u001F",
+				},
+			},
+		},
+		{
+			input:        []byte("not-a-heredoc <<\n"),
+			expectErr:    true,
+			errorMessage: "missing opening heredoc marker on line #1; must contain only alpha-numeric characters, dashes and underscores; got empty string",
+		},
+		{
+			input: []byte(`heredoc <<<EOF
+	content
+	EOF same-line-arg
+	`),
+			expectErr:    true,
+			errorMessage: "too many '<' for heredoc on line #1; only use two, for example <<END",
+		},
+		{
+			input: []byte(`heredoc <<EOF
+	content
+	`),
+			expectErr:    true,
+			errorMessage: "incomplete heredoc <<EOF on line #3, expected ending marker EOF",
+		},
+		{
+			input: []byte(`heredoc <<EOF
+	content
+		EOF
+	`),
+			expectErr:    true,
+			errorMessage: "mismatched leading whitespace in heredoc <<EOF on line #2 [\tcontent], expected whitespace [\t\t] to match the closing marker",
+		},
+		{
+			input: []byte(`heredoc <<EOF
+        content
+		EOF
+	`),
+			expectErr:    true,
+			errorMessage: "mismatched leading whitespace in heredoc <<EOF on line #2 [        content], expected whitespace [\t\t] to match the closing marker",
+		},
+		{
+			input: []byte(`heredoc <<EOF
+The next line is a blank line
+
+The previous line is a blank line
+EOF`),
+			expected: []Token{
+				{Line: 1, Text: "heredoc"},
+				{Line: 1, Text: "The next line is a blank line\n\nThe previous line is a blank line"},
+			},
+		},
+		{
+			input: []byte(`heredoc <<EOF
+	One tab indented heredoc with blank next line
+
+	One tab indented heredoc with blank previous line
+	EOF`),
+			expected: []Token{
+				{Line: 1, Text: "heredoc"},
+				{Line: 1, Text: "One tab indented heredoc with blank next line\n\nOne tab indented heredoc with blank previous line"},
+			},
+		},
+		{
+			input: []byte(`heredoc <<EOF
+The next line is a blank line with one tab
+	
+The previous line is a blank line with one tab
+EOF`),
+			expected: []Token{
+				{Line: 1, Text: "heredoc"},
+				{Line: 1, Text: "The next line is a blank line with one tab\n\t\nThe previous line is a blank line with one tab"},
+			},
+		},
+		{
+			input: []byte(`heredoc <<EOF
+		The next line is a blank line with one tab less than the correct indentation
+	
+		The previous line is a blank line with one tab less than the correct indentation
+		EOF`),
+			expectErr:    true,
+			errorMessage: "mismatched leading whitespace in heredoc <<EOF on line #3 [\t], expected whitespace [\t\t] to match the closing marker",
+		},
 	}
 
 	for i, testCase := range testCases {
 		actual, err := Tokenize(testCase.input, "")
+		if testCase.expectErr {
+			if err == nil {
+				t.Fatalf("expected error, got actual: %v", actual)
+				continue
+			}
+			if err.Error() != testCase.errorMessage {
+				t.Fatalf("expected error '%v', got: %v", testCase.errorMessage, err)
+			}
+			continue
+		}
+
 		if err != nil {
-			t.Errorf("%v", err)
+			t.Fatalf("%v", err)
 		}
 		lexerCompare(t, i, testCase.expected, actual)
 	}
@@ -262,17 +523,17 @@ func TestLexer(t *testing.T) {
 
 func lexerCompare(t *testing.T, n int, expected, actual []Token) {
 	if len(expected) != len(actual) {
-		t.Errorf("Test case %d: expected %d token(s) but got %d", n, len(expected), len(actual))
+		t.Fatalf("Test case %d: expected %d token(s) but got %d", n, len(expected), len(actual))
 	}
 
 	for i := 0; i < len(actual) && i < len(expected); i++ {
 		if actual[i].Line != expected[i].Line {
-			t.Errorf("Test case %d token %d ('%s'): expected line %d but was line %d",
+			t.Fatalf("Test case %d token %d ('%s'): expected line %d but was line %d",
 				n, i, expected[i].Text, expected[i].Line, actual[i].Line)
 			break
 		}
 		if actual[i].Text != expected[i].Text {
-			t.Errorf("Test case %d token %d: expected text '%s' but was '%s'",
+			t.Fatalf("Test case %d token %d: expected text '%s' but was '%s'",
 				n, i, expected[i].Text, actual[i].Text)
 			break
 		}

caddyconfig/caddyfile/parse.go

@@ -20,11 +20,11 @@ import (
 	"io"
 	"os"
 	"path/filepath"
-	"strconv"
 	"strings"
 
-	"github.com/caddyserver/caddy/v2"
 	"go.uber.org/zap"
+
+	"github.com/caddyserver/caddy/v2"
 )
 
 // Parse parses the input just enough to group tokens, in
@@ -50,7 +50,7 @@ func Parse(filename string, input []byte) ([]ServerBlock, error) {
 	p := parser{
 		Dispenser: NewDispenser(tokens),
 		importGraph: importGraph{
-			nodes: make(map[string]bool),
+			nodes: make(map[string]struct{}),
 			edges: make(adjacency),
 		},
 	}
@@ -61,20 +61,12 @@ func Parse(filename string, input []byte) ([]ServerBlock, error) {
 // It returns all the tokens from the input, unstructured
 // and in order. It may mutate input as it expands env vars.
 func allTokens(filename string, input []byte) ([]Token, error) {
-	inputCopy, err := replaceEnvVars(input)
-	if err != nil {
-		return nil, err
-	}
-	tokens, err := Tokenize(inputCopy, filename)
-	if err != nil {
-		return nil, err
-	}
-	return tokens, nil
+	return Tokenize(replaceEnvVars(input), filename)
 }
 
 // replaceEnvVars replaces all occurrences of environment variables.
 // It mutates the underlying array and returns the updated slice.
-func replaceEnvVars(input []byte) ([]byte, error) {
+func replaceEnvVars(input []byte) []byte {
 	var offset int
 	for {
 		begin := bytes.Index(input[offset:], spanOpen)
@@ -115,7 +107,7 @@ func replaceEnvVars(input []byte) ([]byte, error) {
 		// continue at the end of the replacement
 		offset = begin + len(envVarBytes)
 	}
-	return input, nil
+	return input
 }
 
 type parser struct {
@@ -157,7 +149,6 @@ func (p *parser) begin() error {
 	}
 
 	err := p.addresses()
-
 	if err != nil {
 		return err
 	}
@@ -168,6 +159,25 @@ func (p *parser) begin() error {
 		return nil
 	}
 
+	if ok, name := p.isNamedRoute(); ok {
+		// we just need a dummy leading token to ease parsing later
+		nameToken := p.Token()
+		nameToken.Text = name
+
+		// named routes only have one key, the route name
+		p.block.Keys = []Token{nameToken}
+		p.block.IsNamedRoute = true
+
+		// get all the tokens from the block, including the braces
+		tokens, err := p.blockTokens(true)
+		if err != nil {
+			return err
+		}
+		tokens = append([]Token{nameToken}, tokens...)
+		p.block.Segments = []Segment{tokens}
+		return nil
+	}
+
 	if ok, name := p.isSnippet(); ok {
 		if p.definedSnippets == nil {
 			p.definedSnippets = map[string][]Token{}
@@ -176,16 +186,15 @@ func (p *parser) begin() error {
 			return p.Errf("redeclaration of previously declared snippet %s", name)
 		}
 		// consume all tokens til matched close brace
-		tokens, err := p.snippetTokens()
+		tokens, err := p.blockTokens(false)
 		if err != nil {
 			return err
 		}
 		// Just as we need to track which file the token comes from, we need to
-		// keep track of which snippets do the tokens come from. This is helpful
-		// in tracking import cycles across files/snippets by namespacing them. Without
-		// this we end up with false-positives in cycle-detection.
+		// keep track of which snippet the token comes from. This is helpful
+		// in tracking import cycles across files/snippets by namespacing them.
+		// Without this, we end up with false-positives in cycle-detection.
 		for k, v := range tokens {
-			v.inSnippet = true
 			v.snippetName = name
 			tokens[k] = v
 		}
@@ -202,11 +211,17 @@ func (p *parser) addresses() error {
 	var expectingAnother bool
 
 	for {
-		tkn := p.Val()
+		value := p.Val()
+		token := p.Token()
 
-		// special case: import directive replaces tokens during parse-time
-		if tkn == "import" && p.isNewLine() {
-			err := p.doImport()
+		// Reject request matchers if trying to define them globally
+		if strings.HasPrefix(value, "@") {
+			return p.Errf("request matchers may not be defined globally, they must be in a site block; found %s", value)
+		}
+
+		// Special case: import directive replaces tokens during parse-time
+		if value == "import" && p.isNewLine() {
+			err := p.doImport(0)
 			if err != nil {
 				return err
 			}
@@ -214,9 +229,9 @@ func (p *parser) addresses() error {
 		}
 
 		// Open brace definitely indicates end of addresses
-		if tkn == "{" {
+		if value == "{" {
 			if expectingAnother {
-				return p.Errf("Expected another address but had '%s' - check for extra comma", tkn)
+				return p.Errf("Expected another address but had '%s' - check for extra comma", value)
 			}
 			// Mark this server block as being defined with braces.
 			// This is used to provide a better error message when
@@ -228,15 +243,15 @@ func (p *parser) addresses() error {
 		}
 
 		// Users commonly forget to place a space between the address and the '{'
-		if strings.HasSuffix(tkn, "{") {
-			return p.Errf("Site addresses cannot end with a curly brace: '%s' - put a space between the token and the brace", tkn)
+		if strings.HasSuffix(value, "{") {
+			return p.Errf("Site addresses cannot end with a curly brace: '%s' - put a space between the token and the brace", value)
 		}
 
-		if tkn != "" { // empty token possible if user typed ""
+		if value != "" { // empty token possible if user typed ""
 			// Trailing comma indicates another address will follow, which
 			// may possibly be on the next line
-			if tkn[len(tkn)-1] == ',' {
-				tkn = tkn[:len(tkn)-1]
+			if value[len(value)-1] == ',' {
+				value = value[:len(value)-1]
 				expectingAnother = true
 			} else {
 				expectingAnother = false // but we may still see another one on this line
@@ -245,11 +260,17 @@ func (p *parser) addresses() error {
 			// If there's a comma here, it's probably because they didn't use a space
 			// between their two domains, e.g. "foo.com,bar.com", which would not be
 			// parsed as two separate site addresses.
-			if strings.Contains(tkn, ",") {
-				return p.Errf("Site addresses cannot contain a comma ',': '%s' - put a space after the comma to separate site addresses", tkn)
+			if strings.Contains(value, ",") {
+				return p.Errf("Site addresses cannot contain a comma ',': '%s' - put a space after the comma to separate site addresses", value)
 			}
 
-			p.block.Keys = append(p.block.Keys, tkn)
+			// After the above, a comma surrounded by spaces would result
+			// in an empty token which we should ignore
+			if value != "" {
+				// Add the token as a site address
+				token.Text = value
+				p.block.Keys = append(p.block.Keys, token)
+			}
 		}
 
 		// Advance token and possibly break out of loop or return error
@@ -306,7 +327,7 @@ func (p *parser) directives() error {
 
 		// special case: import directive replaces tokens during parse-time
 		if p.Val() == "import" {
-			err := p.doImport()
+			err := p.doImport(1)
 			if err != nil {
 				return err
 			}
@@ -332,7 +353,7 @@ func (p *parser) directives() error {
 // is on the token before where the import directive was. In
 // other words, call Next() to access the first token that was
 // imported.
-func (p *parser) doImport() error {
+func (p *parser) doImport(nesting int) error {
 	// syntax checks
 	if !p.NextArg() {
 		return p.ArgErr()
@@ -345,15 +366,48 @@ func (p *parser) doImport() error {
 	// grab remaining args as placeholder replacements
 	args := p.RemainingArgs()
 
-	// add args to the replacer
-	repl := caddy.NewEmptyReplacer()
-	for index, arg := range args {
-		repl.Set("args."+strconv.Itoa(index), arg)
+	// set up a replacer for non-variadic args replacement
+	repl := makeArgsReplacer(args)
+
+	// grab all the tokens (if it exists) from within a block that follows the import
+	var blockTokens []Token
+	for currentNesting := p.Nesting(); p.NextBlock(currentNesting); {
+		blockTokens = append(blockTokens, p.Token())
+	}
+	// initialize with size 1
+	blockMapping := make(map[string][]Token, 1)
+	if len(blockTokens) > 0 {
+		// use such tokens to create a new dispenser, and then use it to parse each block
+		bd := NewDispenser(blockTokens)
+		for bd.Next() {
+			// see if we can grab a key
+			var currentMappingKey string
+			if bd.Val() == "{" {
+				return p.Err("anonymous blocks are not supported")
+			}
+			currentMappingKey = bd.Val()
+			currentMappingTokens := []Token{}
+			// read all args until end of line / {
+			if bd.NextArg() {
+				currentMappingTokens = append(currentMappingTokens, bd.Token())
+				for bd.NextArg() {
+					currentMappingTokens = append(currentMappingTokens, bd.Token())
+				}
+				// TODO(elee1766): we don't enter another mapping here because it's annoying to extract the { and } properly.
+				// maybe someone can do that in the future
+			} else {
+				// attempt to enter a block and add tokens to the currentMappingTokens
+				for mappingNesting := bd.Nesting(); bd.NextBlock(mappingNesting); {
+					currentMappingTokens = append(currentMappingTokens, bd.Token())
+				}
+			}
+			blockMapping[currentMappingKey] = currentMappingTokens
+		}
 	}
 
 	// splice out the import directive and its arguments
 	// (2 tokens, plus the length of args)
-	tokensBefore := p.tokens[:p.cursor-1-len(args)]
+	tokensBefore := p.tokens[:p.cursor-1-len(args)-len(blockTokens)]
 	tokensAfter := p.tokens[p.cursor+1:]
 	var importedTokens []Token
 	var nodes []string
@@ -369,7 +423,7 @@ func (p *parser) doImport() error {
 		// make path relative to the file of the _token_ being processed rather
 		// than current working directory (issue #867) and then use glob to get
 		// list of matching filenames
-		absFile, err := filepath.Abs(p.Dispenser.File())
+		absFile, err := caddy.FastAbs(p.Dispenser.File())
 		if err != nil {
 			return p.Errf("Failed to get absolute path of file: %s: %v", p.Dispenser.File(), err)
 		}
@@ -387,7 +441,6 @@ func (p *parser) doImport() error {
 			return p.Errf("Glob pattern may only contain one wildcard (*), but has others: %s", globPattern)
 		}
 		matches, err = filepath.Glob(globPattern)
-
 		if err != nil {
 			return p.Errf("Failed to use import pattern %s: %v", importPattern, err)
 		}
@@ -397,6 +450,20 @@ func (p *parser) doImport() error {
 			} else {
 				return p.Errf("File to import not found: %s", importPattern)
 			}
+		} else {
+			// See issue #5295 - should skip any files that start with a . when iterating over them.
+			sep := string(filepath.Separator)
+			segGlobPattern := strings.Split(globPattern, sep)
+			if strings.HasPrefix(segGlobPattern[len(segGlobPattern)-1], "*") {
+				var tmpMatches []string
+				for _, m := range matches {
+					seg := strings.Split(m, sep)
+					if !strings.HasPrefix(seg[len(seg)-1], ".") {
+						tmpMatches = append(tmpMatches, m)
+					}
+				}
+				matches = tmpMatches
+			}
 		}
 
 		// collect all the imported tokens
@@ -411,7 +478,7 @@ func (p *parser) doImport() error {
 	}
 
 	nodeName := p.File()
-	if p.Token().inSnippet {
+	if p.Token().snippetName != "" {
 		nodeName += fmt.Sprintf(":%s", p.Token().snippetName)
 	}
 	p.importGraph.addNode(nodeName)
@@ -422,19 +489,102 @@ func (p *parser) doImport() error {
 	}
 
 	// copy the tokens so we don't overwrite p.definedSnippets
-	tokensCopy := make([]Token, len(importedTokens))
-	copy(tokensCopy, importedTokens)
+	tokensCopy := make([]Token, 0, len(importedTokens))
+
+	var (
+		maybeSnippet   bool
+		maybeSnippetId bool
+		index          int
+	)
 
 	// run the argument replacer on the tokens
-	for index, token := range tokensCopy {
-		token.Text = repl.ReplaceKnown(token.Text, "")
-		tokensCopy[index] = token
+	// golang for range slice return a copy of value
+	// similarly, append also copy value
+	for i, token := range importedTokens {
+		// update the token's imports to refer to import directive filename, line number and snippet name if there is one
+		if token.snippetName != "" {
+			token.imports = append(token.imports, fmt.Sprintf("%s:%d (import %s)", p.File(), p.Line(), token.snippetName))
+		} else {
+			token.imports = append(token.imports, fmt.Sprintf("%s:%d (import)", p.File(), p.Line()))
+		}
+
+		// naive way of determine snippets, as snippets definition can only follow name + block
+		// format, won't check for nesting correctness or any other error, that's what parser does.
+		if !maybeSnippet && nesting == 0 {
+			// first of the line
+			if i == 0 || isNextOnNewLine(tokensCopy[i-1], token) {
+				index = 0
+			} else {
+				index++
+			}
+
+			if index == 0 && len(token.Text) >= 3 && strings.HasPrefix(token.Text, "(") && strings.HasSuffix(token.Text, ")") {
+				maybeSnippetId = true
+			}
+		}
+
+		switch token.Text {
+		case "{":
+			nesting++
+			if index == 1 && maybeSnippetId && nesting == 1 {
+				maybeSnippet = true
+				maybeSnippetId = false
+			}
+		case "}":
+			nesting--
+			if nesting == 0 && maybeSnippet {
+				maybeSnippet = false
+			}
+		}
+		// if it is {block}, we substitute with all tokens in the block
+		// if it is {blocks.*}, we substitute with the tokens in the mapping for the *
+		var skip bool
+		var tokensToAdd []Token
+		switch {
+		case token.Text == "{block}":
+			tokensToAdd = blockTokens
+		case strings.HasPrefix(token.Text, "{blocks.") && strings.HasSuffix(token.Text, "}"):
+			// {blocks.foo.bar} will be extracted to key `foo.bar`
+			blockKey := strings.TrimPrefix(strings.TrimSuffix(token.Text, "}"), "{blocks.")
+			val, ok := blockMapping[blockKey]
+			if ok {
+				tokensToAdd = val
+			}
+		default:
+			skip = true
+		}
+		if !skip {
+			if len(tokensToAdd) == 0 {
+				// if there is no content in the snippet block, don't do any replacement
+				// this allows snippets which contained {block}/{block.*} before this change to continue functioning as normal
+				tokensCopy = append(tokensCopy, token)
+			} else {
+				tokensCopy = append(tokensCopy, tokensToAdd...)
+			}
+			continue
+		}
+
+		if maybeSnippet {
+			tokensCopy = append(tokensCopy, token)
+			continue
+		}
+
+		foundVariadic, startIndex, endIndex := parseVariadic(token, len(args))
+		if foundVariadic {
+			for _, arg := range args[startIndex:endIndex] {
+				token.Text = arg
+				tokensCopy = append(tokensCopy, token)
+			}
+		} else {
+			token.Text = repl.ReplaceKnown(token.Text, "")
+			tokensCopy = append(tokensCopy, token)
+		}
 	}
 
 	// splice the imported tokens in the place of the import statement
 	// and rewind cursor so Next() will land on first imported token
 	p.tokens = append(tokensBefore, append(tokensCopy, tokensAfter...)...)
-	p.cursor -= len(args) + 1
+	p.cursor -= len(args) + len(blockTokens) + 1
 
 	return nil
 }
@@ -459,6 +609,12 @@ func (p *parser) doSingleImport(importFile string) ([]Token, error) {
 		return nil, p.Errf("Could not read imported file %s: %v", importFile, err)
 	}
 
+	// only warning in case of empty files
+	if len(input) == 0 || len(strings.TrimSpace(string(input))) == 0 {
+		caddy.Log().Warn("Import file is empty", zap.String("file", importFile))
+		return []Token{}, nil
+	}
+
 	importedTokens, err := allTokens(importFile, input)
 	if err != nil {
 		return nil, p.Errf("Could not read tokens while importing %s: %v", importFile, err)
@@ -466,7 +622,7 @@ func (p *parser) doSingleImport(importFile string) ([]Token, error) {
 
 	// Tack the file path onto these tokens so errors show the imported file's name
 	// (we use full, absolute path to avoid bugs: issue #1892)
-	filename, err := filepath.Abs(importFile)
+	filename, err := caddy.FastAbs(importFile)
 	if err != nil {
 		return nil, p.Errf("Failed to get absolute path of file: %s: %v", importFile, err)
 	}
@@ -484,7 +640,6 @@ func (p *parser) doSingleImport(importFile string) ([]Token, error) {
 // are loaded into the current server block for later use
 // by directive setup functions.
 func (p *parser) directive() error {
-
 	// a segment is a list of tokens associated with this directive
 	var segment Segment
 
@@ -497,6 +652,9 @@ func (p *parser) directive() error {
 			if !p.isNextOnNewLine() && p.Token().wasQuoted == 0 {
 				return p.Err("Unexpected next token after '{' on same line")
 			}
+			if p.isNewLine() {
+				return p.Err("Unexpected '{' on a new line; did you mean to place the '{' on the previous line?")
+			}
 		} else if p.Val() == "{}" {
 			if p.isNextOnNewLine() && p.Token().wasQuoted == 0 {
 				return p.Err("Unexpected '{}' at end of line")
@@ -509,7 +667,7 @@ func (p *parser) directive() error {
 		} else if p.Val() == "}" && p.nesting == 0 {
 			return p.Err("Unexpected '}' because no matching opening brace")
 		} else if p.Val() == "import" && p.isNewLine() {
-			if err := p.doImport(); err != nil {
+			if err := p.doImport(1); err != nil {
 				return err
 			}
 			p.cursor-- // cursor is advanced when we continue, so roll back one more
@@ -550,28 +708,43 @@ func (p *parser) closeCurlyBrace() error {
 	return nil
 }
 
+func (p *parser) isNamedRoute() (bool, string) {
+	keys := p.block.Keys
+	// A named route block is a single key with parens, prefixed with &.
+	if len(keys) == 1 && strings.HasPrefix(keys[0].Text, "&(") && strings.HasSuffix(keys[0].Text, ")") {
+		return true, strings.TrimSuffix(keys[0].Text[2:], ")")
+	}
+	return false, ""
+}
+
 func (p *parser) isSnippet() (bool, string) {
 	keys := p.block.Keys
 	// A snippet block is a single key with parens. Nothing else qualifies.
-	if len(keys) == 1 && strings.HasPrefix(keys[0], "(") && strings.HasSuffix(keys[0], ")") {
-		return true, strings.TrimSuffix(keys[0][1:], ")")
+	if len(keys) == 1 && strings.HasPrefix(keys[0].Text, "(") && strings.HasSuffix(keys[0].Text, ")") {
+		return true, strings.TrimSuffix(keys[0].Text[1:], ")")
 	}
 	return false, ""
 }
 
 // read and store everything in a block for later replay.
-func (p *parser) snippetTokens() ([]Token, error) {
-	// snippet must have curlies.
+func (p *parser) blockTokens(retainCurlies bool) ([]Token, error) {
+	// block must have curlies.
 	err := p.openCurlyBrace()
 	if err != nil {
 		return nil, err
 	}
-	nesting := 1 // count our own nesting in snippets
+	nesting := 1 // count our own nesting
 	tokens := []Token{}
+	if retainCurlies {
+		tokens = append(tokens, p.Token())
+	}
 	for p.Next() {
 		if p.Val() == "}" {
 			nesting--
 			if nesting == 0 {
+				if retainCurlies {
+					tokens = append(tokens, p.Token())
+				}
 				break
 			}
 		}
@@ -591,9 +764,18 @@ func (p *parser) snippetTokens() ([]Token, error) {
 // head of the server block with tokens, which are
 // grouped by segments.
 type ServerBlock struct {
-	HasBraces bool
-	Keys      []string
-	Segments  []Segment
+	HasBraces    bool
+	Keys         []Token
+	Segments     []Segment
+	IsNamedRoute bool
+}
+
+func (sb ServerBlock) GetKeysText() []string {
+	res := []string{}
+	for _, k := range sb.Keys {
+		res = append(res, k.Text)
+	}
+	return res
 }
 
 // DispenseDirective returns a dispenser that contains

caddyconfig/caddyfile/parse_test.go

@@ -21,11 +21,96 @@ import (
 	"testing"
 )
 
+func TestParseVariadic(t *testing.T) {
+	args := make([]string, 10)
+	for i, tc := range []struct {
+		input  string
+		result bool
+	}{
+		{
+			input:  "",
+			result: false,
+		},
+		{
+			input:  "{args[1",
+			result: false,
+		},
+		{
+			input:  "1]}",
+			result: false,
+		},
+		{
+			input:  "{args[:]}aaaaa",
+			result: false,
+		},
+		{
+			input:  "aaaaa{args[:]}",
+			result: false,
+		},
+		{
+			input:  "{args.}",
+			result: false,
+		},
+		{
+			input:  "{args.1}",
+			result: false,
+		},
+		{
+			input:  "{args[]}",
+			result: false,
+		},
+		{
+			input:  "{args[:]}",
+			result: true,
+		},
+		{
+			input:  "{args[:]}",
+			result: true,
+		},
+		{
+			input:  "{args[0:]}",
+			result: true,
+		},
+		{
+			input:  "{args[:0]}",
+			result: true,
+		},
+		{
+			input:  "{args[-1:]}",
+			result: false,
+		},
+		{
+			input:  "{args[:11]}",
+			result: false,
+		},
+		{
+			input:  "{args[10:0]}",
+			result: false,
+		},
+		{
+			input:  "{args[0:10]}",
+			result: true,
+		},
+		{
+			input:  "{args[0]}:{args[1]}:{args[2]}",
+			result: false,
+		},
+	} {
+		token := Token{
+			File: "test",
+			Line: 1,
+			Text: tc.input,
+		}
+		if v, _, _ := parseVariadic(token, len(args)); v != tc.result {
+			t.Errorf("Test %d error expectation failed Expected: %t, got %t", i, tc.result, v)
+		}
+	}
+}
+
 func TestAllTokens(t *testing.T) {
 	input := []byte("a b c\nd e")
 	expected := []string{"a", "b", "c", "d", "e"}
 	tokens, err := allTokens("TestAllTokens", input)
-
 	if err != nil {
 		t.Fatalf("Expected no error, got %v", err)
 	}
@@ -63,10 +148,11 @@ func TestParseOneAndImport(t *testing.T) {
 			"localhost",
 		}, []int{1}},
 
-		{`localhost:1234
+		{
+			`localhost:1234
 		  dir1 foo bar`, false, []string{
-			"localhost:1234",
-		}, []int{3},
+				"localhost:1234",
+			}, []int{3},
 		},
 
 		{`localhost {
@@ -187,6 +273,23 @@ func TestParseOneAndImport(t *testing.T) {
 
 		{`import testdata/not_found.txt`, true, []string{}, []int{}},
 
+		// empty file should just log a warning, and result in no tokens
+		{`import testdata/empty.txt`, false, []string{}, []int{}},
+
+		{`import testdata/only_white_space.txt`, false, []string{}, []int{}},
+
+		// import path/to/dir/* should skip any files that start with a . when iterating over them.
+		{`localhost
+		  dir1 arg1
+		  import testdata/glob/*`, false, []string{
+			"localhost",
+		}, []int{2, 3, 1}},
+
+		// import path/to/dir/.* should continue to read all dotfiles in a dir.
+		{`import testdata/glob/.*`, false, []string{
+			"host1",
+		}, []int{1, 2}},
+
 		{`""`, false, []string{}, []int{}},
 
 		{``, false, []string{}, []int{}},
@@ -194,6 +297,14 @@ func TestParseOneAndImport(t *testing.T) {
 		// Unexpected next token after '{' on same line
 		{`localhost
 		  dir1 { a b }`, true, []string{"localhost"}, []int{}},
+
+		// Unexpected '{' on a new line
+		{`localhost
+		dir1
+		{
+			a b
+		}`, true, []string{"localhost"}, []int{}},
+
 		// Workaround with quotes
 		{`localhost
 		  dir1 "{" a b "}"`, false, []string{"localhost"}, []int{5}},
@@ -236,7 +347,7 @@ func TestParseOneAndImport(t *testing.T) {
 				i, len(test.keys), len(result.Keys))
 			continue
 		}
-		for j, addr := range result.Keys {
+		for j, addr := range result.GetKeysText() {
 			if addr != test.keys[j] {
 				t.Errorf("Test %d, key %d: Expected '%s', but was '%s'",
 					i, j, test.keys[j], addr)
@@ -268,8 +379,9 @@ func TestRecursiveImport(t *testing.T) {
 	}
 
 	isExpected := func(got ServerBlock) bool {
-		if len(got.Keys) != 1 || got.Keys[0] != "localhost" {
-			t.Errorf("got keys unexpected: expect localhost, got %v", got.Keys)
+		textKeys := got.GetKeysText()
+		if len(textKeys) != 1 || textKeys[0] != "localhost" {
+			t.Errorf("got keys unexpected: expect localhost, got %v", textKeys)
 			return false
 		}
 		if len(got.Segments) != 2 {
@@ -296,13 +408,13 @@ func TestRecursiveImport(t *testing.T) {
 	err = os.WriteFile(recursiveFile1, []byte(
 		`localhost
 		dir1
-		import recursive_import_test2`), 0644)
+		import recursive_import_test2`), 0o644)
 	if err != nil {
 		t.Fatal(err)
 	}
 	defer os.Remove(recursiveFile1)
 
-	err = os.WriteFile(recursiveFile2, []byte("dir2 1"), 0644)
+	err = os.WriteFile(recursiveFile2, []byte("dir2 1"), 0o644)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -330,7 +442,7 @@ func TestRecursiveImport(t *testing.T) {
 	err = os.WriteFile(recursiveFile1, []byte(
 		`localhost
 		dir1
-		import `+recursiveFile2), 0644)
+		import `+recursiveFile2), 0o644)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -363,8 +475,9 @@ func TestDirectiveImport(t *testing.T) {
 	}
 
 	isExpected := func(got ServerBlock) bool {
-		if len(got.Keys) != 1 || got.Keys[0] != "localhost" {
-			t.Errorf("got keys unexpected: expect localhost, got %v", got.Keys)
+		textKeys := got.GetKeysText()
+		if len(textKeys) != 1 || textKeys[0] != "localhost" {
+			t.Errorf("got keys unexpected: expect localhost, got %v", textKeys)
 			return false
 		}
 		if len(got.Segments) != 2 {
@@ -384,7 +497,7 @@ func TestDirectiveImport(t *testing.T) {
 	}
 
 	err = os.WriteFile(directiveFile, []byte(`prop1 1
-	prop2 2`), 0644)
+	prop2 2`), 0o644)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -442,6 +555,10 @@ func TestParseAll(t *testing.T) {
 			{"localhost:1234", "http://host2"},
 		}},
 
+		{`foo.example.com   ,   example.com`, false, [][]string{
+			{"foo.example.com", "example.com"},
+		}},
+
 		{`localhost:1234, http://host2,`, true, [][]string{}},
 
 		{`http://host1.com, http://host2.com {
@@ -501,11 +618,11 @@ func TestParseAll(t *testing.T) {
 		}
 		for j, block := range blocks {
 			if len(block.Keys) != len(test.keys[j]) {
-				t.Errorf("Test %d: Expected %d keys in block %d, got %d",
-					i, len(test.keys[j]), j, len(block.Keys))
+				t.Errorf("Test %d: Expected %d keys in block %d, got %d: %v",
+					i, len(test.keys[j]), j, len(block.Keys), block.Keys)
 				continue
 			}
-			for k, addr := range block.Keys {
+			for k, addr := range block.GetKeysText() {
 				if addr != test.keys[j][k] {
 					t.Errorf("Test %d, block %d, key %d: Expected '%s', but got '%s'",
 						i, j, k, test.keys[j][k], addr)
@@ -604,16 +721,43 @@ func TestEnvironmentReplacement(t *testing.T) {
 			expect: "}{$",
 		},
 	} {
-		actual, err := replaceEnvVars([]byte(test.input))
-		if err != nil {
-			t.Fatal(err)
-		}
+		actual := replaceEnvVars([]byte(test.input))
 		if !bytes.Equal(actual, []byte(test.expect)) {
 			t.Errorf("Test %d: Expected: '%s' but got '%s'", i, test.expect, actual)
 		}
 	}
 }
 
+func TestImportReplacementInJSONWithBrace(t *testing.T) {
+	for i, test := range []struct {
+		args   []string
+		input  string
+		expect string
+	}{
+		{
+			args:   []string{"123"},
+			input:  "{args[0]}",
+			expect: "123",
+		},
+		{
+			args:   []string{"123"},
+			input:  `{"key":"{args[0]}"}`,
+			expect: `{"key":"123"}`,
+		},
+		{
+			args:   []string{"123", "123"},
+			input:  `{"key":[{args[0]},{args[1]}]}`,
+			expect: `{"key":[123,123]}`,
+		},
+	} {
+		repl := makeArgsReplacer(test.args)
+		actual := repl.ReplaceKnown(test.input, "")
+		if actual != test.expect {
+			t.Errorf("Test %d: Expected: '%s' but got '%s'", i, test.expect, actual)
+		}
+	}
+}
+
 func TestSnippets(t *testing.T) {
 	p := testParser(`
 		(common) {
@@ -631,7 +775,7 @@ func TestSnippets(t *testing.T) {
 	if len(blocks) != 1 {
 		t.Fatalf("Expect exactly one server block. Got %d.", len(blocks))
 	}
-	if actual, expected := blocks[0].Keys[0], "http://example.com"; expected != actual {
+	if actual, expected := blocks[0].GetKeysText()[0], "http://example.com"; expected != actual {
 		t.Errorf("Expected server name to be '%s' but was '%s'", expected, actual)
 	}
 	if len(blocks[0].Segments) != 2 {
@@ -663,7 +807,7 @@ func TestImportedFilesIgnoreNonDirectiveImportTokens(t *testing.T) {
 	fileName := writeStringToTempFileOrDie(t, `
 		http://example.com {
 			# This isn't an import directive, it's just an arg with value 'import'
-			basicauth / import password
+			basic_auth / import password
 		}
 	`)
 	// Parse the root file that imports the other one.
@@ -674,12 +818,12 @@ func TestImportedFilesIgnoreNonDirectiveImportTokens(t *testing.T) {
 	}
 	auth := blocks[0].Segments[0]
 	line := auth[0].Text + " " + auth[1].Text + " " + auth[2].Text + " " + auth[3].Text
-	if line != "basicauth / import password" {
+	if line != "basic_auth / import password" {
 		// Previously, it would be changed to:
-		//   basicauth / import /path/to/test/dir/password
+		//   basic_auth / import /path/to/test/dir/password
 		// referencing a file that (probably) doesn't exist and changing the
 		// password!
-		t.Errorf("Expected basicauth tokens to be 'basicauth / import password' but got %#q", line)
+		t.Errorf("Expected basic_auth tokens to be 'basic_auth / import password' but got %#q", line)
 	}
 }
 
@@ -706,7 +850,7 @@ func TestSnippetAcrossMultipleFiles(t *testing.T) {
 	if len(blocks) != 1 {
 		t.Fatalf("Expect exactly one server block. Got %d.", len(blocks))
 	}
-	if actual, expected := blocks[0].Keys[0], "http://example.com"; expected != actual {
+	if actual, expected := blocks[0].GetKeysText()[0], "http://example.com"; expected != actual {
 		t.Errorf("Expected server name to be '%s' but was '%s'", expected, actual)
 	}
 	if len(blocks[0].Segments) != 1 {
@@ -717,6 +861,29 @@ func TestSnippetAcrossMultipleFiles(t *testing.T) {
 	}
 }
 
+func TestRejectsGlobalMatcher(t *testing.T) {
+	p := testParser(`
+		@rejected path /foo
+
+		(common) {
+			gzip foo
+			errors stderr
+		}
+
+		http://example.com {
+			import common
+		}
+	`)
+	_, err := p.parseAll()
+	if err == nil {
+		t.Fatal("Expected an error, but got nil")
+	}
+	expected := "request matchers may not be defined globally, they must be in a site block; found @rejected, at Testfile:2"
+	if err.Error() != expected {
+		t.Errorf("Expected error to be '%s' but got '%v'", expected, err)
+	}
+}
+
 func testParser(input string) parser {
 	return parser{Dispenser: NewTestDispenser(input)}
 }

caddyconfig/caddyfile/testdata/glob/.dotfile.txt

@@ -0,0 +1,4 @@
+host1 {
+	dir1
+	dir2 arg1
+}

caddyconfig/caddyfile/testdata/glob/import_test1.txt

@@ -0,0 +1,2 @@
+dir2 arg1 arg2
+dir3

caddyconfig/caddyfile/testdata/import_args0.txt

@@ -1 +1 @@
-{args.0}
+{args[0]}

caddyconfig/caddyfile/testdata/import_args1.txt

@@ -1 +1 @@
-{args.0} {args.1}
+{args[0]} {args[1]}

caddyconfig/caddyfile/testdata/only_white_space.txt

@@ -0,0 +1,7 @@
+
+
+ 
+  
+　
+
+

caddyconfig/httpcaddyfile/addresses.go

@@ -24,13 +24,14 @@ import (
 	"strings"
 	"unicode"
 
+	"github.com/caddyserver/certmagic"
+
 	"github.com/caddyserver/caddy/v2"
 	"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
 	"github.com/caddyserver/caddy/v2/modules/caddyhttp"
-	"github.com/caddyserver/certmagic"
 )
 
-// mapAddressToServerBlocks returns a map of listener address to list of server
+// mapAddressToProtocolToServerBlocks returns a map of listener address to list of server
 // blocks that will be served on that address. To do this, each server block is
 // expanded so that each one is considered individually, although keys of a
 // server block that share the same address stay grouped together so the config
@@ -76,9 +77,15 @@ import (
 // repetition may be undesirable, so call consolidateAddrMappings() to map
 // multiple addresses to the same lists of server blocks (a many:many mapping).
 // (Doing this is essentially a map-reduce technique.)
-func (st *ServerType) mapAddressToServerBlocks(originalServerBlocks []serverBlock,
-	options map[string]any) (map[string][]serverBlock, error) {
-	sbmap := make(map[string][]serverBlock)
+func (st *ServerType) mapAddressToProtocolToServerBlocks(originalServerBlocks []serverBlock,
+	options map[string]any,
+) (map[string]map[string][]serverBlock, error) {
+	addrToProtocolToServerBlocks := map[string]map[string][]serverBlock{}
+
+	type keyWithParsedKey struct {
+		key       caddyfile.Token
+		parsedKey Address
+	}
 
 	for i, sblock := range originalServerBlocks {
 		// within a server block, we need to map all the listener addresses
@@ -86,27 +93,48 @@ func (st *ServerType) mapAddressToServerBlocks(originalServerBlocks []serverBloc
 		// will be served by them; this has the effect of treating each
 		// key of a server block as its own, but without having to repeat its
 		// contents in cases where multiple keys really can be served together
-		addrToKeys := make(map[string][]string)
+		addrToProtocolToKeyWithParsedKeys := map[string]map[string][]keyWithParsedKey{}
 		for j, key := range sblock.block.Keys {
+			parsedKey, err := ParseAddress(key.Text)
+			if err != nil {
+				return nil, fmt.Errorf("parsing key: %v", err)
+			}
+			parsedKey = parsedKey.Normalize()
+
 			// a key can have multiple listener addresses if there are multiple
 			// arguments to the 'bind' directive (although they will all have
 			// the same port, since the port is defined by the key or is implicit
 			// through automatic HTTPS)
-			addrs, err := st.listenerAddrsForServerBlockKey(sblock, key, options)
+			listeners, err := st.listenersForServerBlockAddress(sblock, parsedKey, options)
 			if err != nil {
-				return nil, fmt.Errorf("server block %d, key %d (%s): determining listener address: %v", i, j, key, err)
+				return nil, fmt.Errorf("server block %d, key %d (%s): determining listener address: %v", i, j, key.Text, err)
 			}
 
-			// associate this key with each listener address it is served on
-			for _, addr := range addrs {
-				addrToKeys[addr] = append(addrToKeys[addr], key)
+			// associate this key with its protocols and each listener address served with them
+			kwpk := keyWithParsedKey{key, parsedKey}
+			for addr, protocols := range listeners {
+				protocolToKeyWithParsedKeys, ok := addrToProtocolToKeyWithParsedKeys[addr]
+				if !ok {
+					protocolToKeyWithParsedKeys = map[string][]keyWithParsedKey{}
+					addrToProtocolToKeyWithParsedKeys[addr] = protocolToKeyWithParsedKeys
+				}
+
+				// an empty protocol indicates the default, a nil or empty value in the ListenProtocols array
+				if len(protocols) == 0 {
+					protocols[""] = struct{}{}
+				}
+				for prot := range protocols {
+					protocolToKeyWithParsedKeys[prot] = append(
+						protocolToKeyWithParsedKeys[prot],
+						kwpk)
+				}
 			}
 		}
 
 		// make a slice of the map keys so we can iterate in sorted order
-		addrs := make([]string, 0, len(addrToKeys))
-		for k := range addrToKeys {
-			addrs = append(addrs, k)
+		addrs := make([]string, 0, len(addrToProtocolToKeyWithParsedKeys))
+		for addr := range addrToProtocolToKeyWithParsedKeys {
+			addrs = append(addrs, addr)
 		}
 		sort.Strings(addrs)
 
@@ -116,83 +144,142 @@ func (st *ServerType) mapAddressToServerBlocks(originalServerBlocks []serverBloc
 		// server block are only the ones which use the address; but
 		// the contents (tokens) are of course the same
 		for _, addr := range addrs {
-			keys := addrToKeys[addr]
-			// parse keys so that we only have to do it once
-			parsedKeys := make([]Address, 0, len(keys))
-			for _, key := range keys {
-				addr, err := ParseAddress(key)
-				if err != nil {
-					return nil, fmt.Errorf("parsing key '%s': %v", key, err)
-				}
-				parsedKeys = append(parsedKeys, addr.Normalize())
+			protocolToKeyWithParsedKeys := addrToProtocolToKeyWithParsedKeys[addr]
+
+			prots := make([]string, 0, len(protocolToKeyWithParsedKeys))
+			for prot := range protocolToKeyWithParsedKeys {
+				prots = append(prots, prot)
 			}
-			sbmap[addr] = append(sbmap[addr], serverBlock{
-				block: caddyfile.ServerBlock{
-					Keys:     keys,
-					Segments: sblock.block.Segments,
-				},
-				pile: sblock.pile,
-				keys: parsedKeys,
+			sort.Strings(prots)
+
+			protocolToServerBlocks, ok := addrToProtocolToServerBlocks[addr]
+			if !ok {
+				protocolToServerBlocks = map[string][]serverBlock{}
+				addrToProtocolToServerBlocks[addr] = protocolToServerBlocks
+			}
+
+			for _, prot := range prots {
+				keyWithParsedKeys := protocolToKeyWithParsedKeys[prot]
+
+				keys := make([]caddyfile.Token, len(keyWithParsedKeys))
+				parsedKeys := make([]Address, len(keyWithParsedKeys))
+
+				for k, keyWithParsedKey := range keyWithParsedKeys {
+					keys[k] = keyWithParsedKey.key
+					parsedKeys[k] = keyWithParsedKey.parsedKey
+				}
+
+				protocolToServerBlocks[prot] = append(protocolToServerBlocks[prot], serverBlock{
+					block: caddyfile.ServerBlock{
+						Keys:     keys,
+						Segments: sblock.block.Segments,
+					},
+					pile:       sblock.pile,
+					parsedKeys: parsedKeys,
+				})
+			}
+		}
+	}
+
+	return addrToProtocolToServerBlocks, nil
+}
+
+// consolidateAddrMappings eliminates repetition of identical server blocks in a mapping of
+// single listener addresses to protocols to lists of server blocks. Since multiple addresses
+// may serve multiple protocols to identical sites (server block contents), this function turns
+// a 1:many mapping into a many:many mapping. Server block contents (tokens) must be
+// exactly identical so that reflect.DeepEqual returns true in order for the addresses to be combined.
+// Identical entries are deleted from the addrToServerBlocks map. Essentially, each pairing (each
+// association from multiple addresses to multiple server blocks; i.e. each element of
+// the returned slice) becomes a server definition in the output JSON.
+func (st *ServerType) consolidateAddrMappings(addrToProtocolToServerBlocks map[string]map[string][]serverBlock) []sbAddrAssociation {
+	sbaddrs := make([]sbAddrAssociation, 0, len(addrToProtocolToServerBlocks))
+
+	addrs := make([]string, 0, len(addrToProtocolToServerBlocks))
+	for addr := range addrToProtocolToServerBlocks {
+		addrs = append(addrs, addr)
+	}
+	sort.Strings(addrs)
+
+	for _, addr := range addrs {
+		protocolToServerBlocks := addrToProtocolToServerBlocks[addr]
+
+		prots := make([]string, 0, len(protocolToServerBlocks))
+		for prot := range protocolToServerBlocks {
+			prots = append(prots, prot)
+		}
+		sort.Strings(prots)
+
+		for _, prot := range prots {
+			serverBlocks := protocolToServerBlocks[prot]
+
+			// now find other addresses that map to identical
+			// server blocks and add them to our map of listener
+			// addresses and protocols, while removing them from
+			// the original map
+			listeners := map[string]map[string]struct{}{}
+
+			for otherAddr, otherProtocolToServerBlocks := range addrToProtocolToServerBlocks {
+				for otherProt, otherServerBlocks := range otherProtocolToServerBlocks {
+					if addr == otherAddr && prot == otherProt || reflect.DeepEqual(serverBlocks, otherServerBlocks) {
+						listener, ok := listeners[otherAddr]
+						if !ok {
+							listener = map[string]struct{}{}
+							listeners[otherAddr] = listener
+						}
+						listener[otherProt] = struct{}{}
+						delete(otherProtocolToServerBlocks, otherProt)
+					}
+				}
+			}
+
+			addresses := make([]string, 0, len(listeners))
+			for lnAddr := range listeners {
+				addresses = append(addresses, lnAddr)
+			}
+			sort.Strings(addresses)
+
+			addressesWithProtocols := make([]addressWithProtocols, 0, len(listeners))
+
+			for _, lnAddr := range addresses {
+				lnProts := listeners[lnAddr]
+				prots := make([]string, 0, len(lnProts))
+				for prot := range lnProts {
+					prots = append(prots, prot)
+				}
+				sort.Strings(prots)
+
+				addressesWithProtocols = append(addressesWithProtocols, addressWithProtocols{
+					address:   lnAddr,
+					protocols: prots,
+				})
+			}
+
+			sbaddrs = append(sbaddrs, sbAddrAssociation{
+				addressesWithProtocols: addressesWithProtocols,
+				serverBlocks:           serverBlocks,
 			})
 		}
 	}
 
-	return sbmap, nil
-}
-
-// consolidateAddrMappings eliminates repetition of identical server blocks in a mapping of
-// single listener addresses to lists of server blocks. Since multiple addresses may serve
-// identical sites (server block contents), this function turns a 1:many mapping into a
-// many:many mapping. Server block contents (tokens) must be exactly identical so that
-// reflect.DeepEqual returns true in order for the addresses to be combined. Identical
-// entries are deleted from the addrToServerBlocks map. Essentially, each pairing (each
-// association from multiple addresses to multiple server blocks; i.e. each element of
-// the returned slice) becomes a server definition in the output JSON.
-func (st *ServerType) consolidateAddrMappings(addrToServerBlocks map[string][]serverBlock) []sbAddrAssociation {
-	sbaddrs := make([]sbAddrAssociation, 0, len(addrToServerBlocks))
-	for addr, sblocks := range addrToServerBlocks {
-		// we start with knowing that at least this address
-		// maps to these server blocks
-		a := sbAddrAssociation{
-			addresses:    []string{addr},
-			serverBlocks: sblocks,
-		}
-
-		// now find other addresses that map to identical
-		// server blocks and add them to our list of
-		// addresses, while removing them from the map
-		for otherAddr, otherSblocks := range addrToServerBlocks {
-			if addr == otherAddr {
-				continue
-			}
-			if reflect.DeepEqual(sblocks, otherSblocks) {
-				a.addresses = append(a.addresses, otherAddr)
-				delete(addrToServerBlocks, otherAddr)
-			}
-		}
-		sort.Strings(a.addresses)
-
-		sbaddrs = append(sbaddrs, a)
-	}
-
-	// sort them by their first address (we know there will always be at least one)
-	// to avoid problems with non-deterministic ordering (makes tests flaky)
-	sort.Slice(sbaddrs, func(i, j int) bool {
-		return sbaddrs[i].addresses[0] < sbaddrs[j].addresses[0]
-	})
-
 	return sbaddrs
 }
 
-// listenerAddrsForServerBlockKey essentially converts the Caddyfile
-// site addresses to Caddy listener addresses for each server block.
-func (st *ServerType) listenerAddrsForServerBlockKey(sblock serverBlock, key string,
-	options map[string]any) ([]string, error) {
-	addr, err := ParseAddress(key)
-	if err != nil {
-		return nil, fmt.Errorf("parsing key: %v", err)
+// listenersForServerBlockAddress essentially converts the Caddyfile site addresses to a map from
+// Caddy listener addresses and the protocols to serve them with to the parsed address for each server block.
+func (st *ServerType) listenersForServerBlockAddress(sblock serverBlock, addr Address,
+	options map[string]any,
+) (map[string]map[string]struct{}, error) {
+	switch addr.Scheme {
+	case "wss":
+		return nil, fmt.Errorf("the scheme wss:// is only supported in browsers; use https:// instead")
+	case "ws":
+		return nil, fmt.Errorf("the scheme ws:// is only supported in browsers; use http:// instead")
+	case "https", "http", "":
+		// Do nothing or handle the valid schemes
+	default:
+		return nil, fmt.Errorf("unsupported URL scheme %s://", addr.Scheme)
 	}
-	addr = addr.Normalize()
 
 	// figure out the HTTP and HTTPS ports; either
 	// use defaults, or override with user config
@@ -216,55 +303,58 @@ func (st *ServerType) listenerAddrsForServerBlockKey(sblock serverBlock, key str
 
 	// error if scheme and port combination violate convention
 	if (addr.Scheme == "http" && lnPort == httpsPort) || (addr.Scheme == "https" && lnPort == httpPort) {
-		return nil, fmt.Errorf("[%s] scheme and port violate convention", key)
+		return nil, fmt.Errorf("[%s] scheme and port violate convention", addr.String())
 	}
 
-	// the bind directive specifies hosts (and potentially network), but is optional
-	lnHosts := make([]string, 0, len(sblock.pile["bind"]))
+	// the bind directive specifies hosts (and potentially network), and the protocols to serve them with, but is optional
+	lnCfgVals := make([]addressesWithProtocols, 0, len(sblock.pile["bind"]))
 	for _, cfgVal := range sblock.pile["bind"] {
-		lnHosts = append(lnHosts, cfgVal.Value.([]string)...)
+		if val, ok := cfgVal.Value.(addressesWithProtocols); ok {
+			lnCfgVals = append(lnCfgVals, val)
+		}
 	}
-	if len(lnHosts) == 0 {
-		if defaultBind, ok := options["default_bind"].([]string); ok {
-			lnHosts = defaultBind
+	if len(lnCfgVals) == 0 {
+		if defaultBindValues, ok := options["default_bind"].([]ConfigValue); ok {
+			for _, defaultBindValue := range defaultBindValues {
+				lnCfgVals = append(lnCfgVals, defaultBindValue.Value.(addressesWithProtocols))
+			}
 		} else {
-			lnHosts = []string{""}
+			lnCfgVals = []addressesWithProtocols{{
+				addresses: []string{""},
+				protocols: nil,
+			}}
 		}
 	}
 
 	// use a map to prevent duplication
-	listeners := make(map[string]struct{})
-	for _, lnHost := range lnHosts {
-		// normally we would simply append the port,
-		// but if lnHost is IPv6, we need to ensure it
-		// is enclosed in [ ]; net.JoinHostPort does
-		// this for us, but lnHost might also have a
-		// network type in front (e.g. "tcp/") leading
-		// to "[tcp/::1]" which causes parsing failures
-		// later; what we need is "tcp/[::1]", so we have
-		// to split the network and host, then re-combine
-		network, host, ok := strings.Cut(lnHost, "/")
-		if !ok {
-			host = network
-			network = ""
+	listeners := map[string]map[string]struct{}{}
+	for _, lnCfgVal := range lnCfgVals {
+		for _, lnAddr := range lnCfgVal.addresses {
+			lnNetw, lnHost, _, err := caddy.SplitNetworkAddress(lnAddr)
+			if err != nil {
+				return nil, fmt.Errorf("splitting listener address: %v", err)
+			}
+			networkAddr, err := caddy.ParseNetworkAddress(caddy.JoinNetworkAddress(lnNetw, lnHost, lnPort))
+			if err != nil {
+				return nil, fmt.Errorf("parsing network address: %v", err)
+			}
+			if _, ok := listeners[addr.String()]; !ok {
+				listeners[networkAddr.String()] = map[string]struct{}{}
+			}
+			for _, protocol := range lnCfgVal.protocols {
+				listeners[networkAddr.String()][protocol] = struct{}{}
+			}
 		}
-		host = strings.Trim(host, "[]") // IPv6
-		networkAddr := caddy.JoinNetworkAddress(network, host, lnPort)
-		addr, err := caddy.ParseNetworkAddress(networkAddr)
-		if err != nil {
-			return nil, fmt.Errorf("parsing network address: %v", err)
-		}
-		listeners[addr.String()] = struct{}{}
 	}
 
-	// now turn map into list
-	listenersList := make([]string, 0, len(listeners))
-	for lnStr := range listeners {
-		listenersList = append(listenersList, lnStr)
-	}
-	sort.Strings(listenersList)
+	return listeners, nil
+}
 
-	return listenersList, nil
+// addressesWithProtocols associates a list of listen addresses
+// with a list of protocols to serve them with
+type addressesWithProtocols struct {
+	addresses []string
+	protocols []string
 }
 
 // Address represents a site address. It contains

caddyconfig/httpcaddyfile/builtins_test.go

@@ -1,6 +1,7 @@
 package httpcaddyfile
 
 import (
+	"strings"
 	"testing"
 
 	"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
@@ -24,11 +25,12 @@ func TestLogDirectiveSyntax(t *testing.T) {
 		{
 			input: `:8080 {
 				log {
+					core mock
 					output file foo.log
 				}
 			}
 			`,
-			output:      `{"logging":{"logs":{"default":{"exclude":["http.log.access.log0"]},"log0":{"writer":{"filename":"foo.log","output":"file"},"include":["http.log.access.log0"]}}},"apps":{"http":{"servers":{"srv0":{"listen":[":8080"],"logs":{"default_logger_name":"log0"}}}}}}`,
+			output:      `{"logging":{"logs":{"default":{"exclude":["http.log.access.log0"]},"log0":{"writer":{"filename":"foo.log","output":"file"},"core":{"module":"mock"},"include":["http.log.access.log0"]}}},"apps":{"http":{"servers":{"srv0":{"listen":[":8080"],"logs":{"default_logger_name":"log0"}}}}}}`,
 			expectError: false,
 		},
 		{
@@ -51,12 +53,28 @@ func TestLogDirectiveSyntax(t *testing.T) {
 		},
 		{
 			input: `:8080 {
-				log invalid {
+				log name-override {
+					core mock
 					output file foo.log
 				}
 			}
 			`,
-			expectError: true,
+			output:      `{"logging":{"logs":{"default":{"exclude":["http.log.access.name-override"]},"name-override":{"writer":{"filename":"foo.log","output":"file"},"core":{"module":"mock"},"include":["http.log.access.name-override"]}}},"apps":{"http":{"servers":{"srv0":{"listen":[":8080"],"logs":{"default_logger_name":"name-override"}}}}}}`,
+			expectError: false,
+		},
+		{
+			input: `:8080 {
+				log {
+					sampling {
+						interval 2
+						first 3
+						thereafter 4
+					}
+				}
+			}
+			`,
+			output:      `{"logging":{"logs":{"default":{"exclude":["http.log.access.log0"]},"log0":{"sampling":{"interval":2,"first":3,"thereafter":4},"include":["http.log.access.log0"]}}},"apps":{"http":{"servers":{"srv0":{"listen":[":8080"],"logs":{"default_logger_name":"log0"}}}}}}`,
+			expectError: false,
 		},
 	} {
 
@@ -213,3 +231,139 @@ func TestRedirDirectiveSyntax(t *testing.T) {
 		}
 	}
 }
+
+func TestImportErrorLine(t *testing.T) {
+	for i, tc := range []struct {
+		input     string
+		errorFunc func(err error) bool
+	}{
+		{
+			input: `(t1) {
+					abort {args[:]}
+				}
+				:8080 {
+					import t1
+					import t1 true
+				}`,
+			errorFunc: func(err error) bool {
+				return err != nil && strings.Contains(err.Error(), "Caddyfile:6 (import t1)")
+			},
+		},
+		{
+			input: `(t1) {
+					abort {args[:]}
+				}
+				:8080 {
+					import t1 true
+				}`,
+			errorFunc: func(err error) bool {
+				return err != nil && strings.Contains(err.Error(), "Caddyfile:5 (import t1)")
+			},
+		},
+		{
+			input: `
+				import testdata/import_variadic_snippet.txt
+				:8080 {
+					import t1 true
+				}`,
+			errorFunc: func(err error) bool {
+				return err == nil
+			},
+		},
+		{
+			input: `
+				import testdata/import_variadic_with_import.txt
+				:8080 {
+					import t1 true
+					import t2 true
+				}`,
+			errorFunc: func(err error) bool {
+				return err == nil
+			},
+		},
+	} {
+		adapter := caddyfile.Adapter{
+			ServerType: ServerType{},
+		}
+
+		_, _, err := adapter.Adapt([]byte(tc.input), nil)
+
+		if !tc.errorFunc(err) {
+			t.Errorf("Test %d error expectation failed, got %s", i, err)
+			continue
+		}
+	}
+}
+
+func TestNestedImport(t *testing.T) {
+	for i, tc := range []struct {
+		input     string
+		errorFunc func(err error) bool
+	}{
+		{
+			input: `(t1) {
+						respond {args[0]} {args[1]}
+					}
+					
+					(t2) {
+						import t1 {args[0]} 202
+					}
+					
+					:8080 {
+						handle {
+							import t2 "foobar"
+						}
+					}`,
+			errorFunc: func(err error) bool {
+				return err == nil
+			},
+		},
+		{
+			input: `(t1) {
+						respond {args[:]}
+					}
+					
+					(t2) {
+						import t1 {args[0]} {args[1]}
+					}
+					
+					:8080 {
+						handle {
+							import t2 "foobar" 202
+						}
+					}`,
+			errorFunc: func(err error) bool {
+				return err == nil
+			},
+		},
+		{
+			input: `(t1) {
+						respond {args[0]} {args[1]}
+					}
+					
+					(t2) {
+						import t1 {args[:]}
+					}
+					
+					:8080 {
+						handle {
+							import t2 "foobar" 202
+						}
+					}`,
+			errorFunc: func(err error) bool {
+				return err == nil
+			},
+		},
+	} {
+		adapter := caddyfile.Adapter{
+			ServerType: ServerType{},
+		}
+
+		_, _, err := adapter.Adapt([]byte(tc.input), nil)
+
+		if !tc.errorFunc(err) {
+			t.Errorf("Test %d error expectation failed, got %s", i, err)
+			continue
+		}
+	}
+}

caddyconfig/httpcaddyfile/directives.go

@@ -16,7 +16,9 @@ package httpcaddyfile
 
 import (
 	"encoding/json"
+	"maps"
 	"net"
+	"slices"
 	"sort"
 	"strconv"
 	"strings"
@@ -27,22 +29,33 @@ import (
 	"github.com/caddyserver/caddy/v2/modules/caddyhttp"
 )
 
-// directiveOrder specifies the order
-// to apply directives in HTTP routes.
+// defaultDirectiveOrder specifies the default order
+// to apply directives in HTTP routes. This must only
+// consist of directives that are included in Caddy's
+// standard distribution.
 //
-// The root directive goes first in case rewrites or
-// redirects depend on existence of files, i.e. the
-// file matcher, which must know the root first.
+// e.g. The 'root' directive goes near the start in
+// case rewrites or redirects depend on existence of
+// files, i.e. the file matcher, which must know the
+// root first.
 //
-// The header directive goes second so that headers
-// can be manipulated before doing redirects.
-var directiveOrder = []string{
+// e.g. The 'header' directive goes before 'redir' so
+// that headers can be manipulated before doing redirects.
+//
+// e.g. The 'respond' directive is near the end because it
+// writes a response and terminates the middleware chain.
+var defaultDirectiveOrder = []string{
 	"tracing",
 
+	// set variables that may be used by other directives
 	"map",
 	"vars",
+	"fs",
 	"root",
-	"skip_log",
+	"log_append",
+	"skip_log", // TODO: deprecated, renamed to log_skip
+	"log_skip",
+	"log_name",
 
 	"header",
 	"copy_response_headers", // only in reverse_proxy's handle_response
@@ -57,14 +70,17 @@ var directiveOrder = []string{
 	"try_files",
 
 	// middleware handlers; some wrap responses
-	"basicauth",
+	"basicauth", // TODO: deprecated, renamed to basic_auth
+	"basic_auth",
 	"forward_auth",
 	"request_header",
 	"encode",
 	"push",
+	"intercept",
 	"templates",
 
 	// special routing & dispatching directives
+	"invoke",
 	"handle",
 	"handle_path",
 	"route",
@@ -81,16 +97,10 @@ var directiveOrder = []string{
 	"acme_server",
 }
 
-// directiveIsOrdered returns true if dir is
-// a known, ordered (sorted) directive.
-func directiveIsOrdered(dir string) bool {
-	for _, d := range directiveOrder {
-		if d == dir {
-			return true
-		}
-	}
-	return false
-}
+// directiveOrder specifies the order to apply directives
+// in HTTP routes, after being modified by either the
+// plugins or by the user via the "order" global option.
+var directiveOrder = defaultDirectiveOrder
 
 // RegisterDirective registers a unique directive dir with an
 // associated unmarshaling (setup) function. When directive dir
@@ -127,6 +137,55 @@ func RegisterHandlerDirective(dir string, setupFunc UnmarshalHandlerFunc) {
 	})
 }
 
+// RegisterDirectiveOrder registers the default order for a
+// directive from a plugin.
+//
+// This is useful when a plugin has a well-understood place
+// it should run in the middleware pipeline, and it allows
+// users to avoid having to define the order themselves.
+//
+// The directive dir may be placed in the position relative
+// to ('before' or 'after') a directive included in Caddy's
+// standard distribution. It cannot be relative to another
+// plugin's directive.
+//
+// EXPERIMENTAL: This API may change or be removed.
+func RegisterDirectiveOrder(dir string, position Positional, standardDir string) {
+	// check if directive was already ordered
+	if slices.Contains(directiveOrder, dir) {
+		panic("directive '" + dir + "' already ordered")
+	}
+
+	if position != Before && position != After {
+		panic("the 2nd argument must be either 'before' or 'after', got '" + position + "'")
+	}
+
+	// check if directive exists in standard distribution, since
+	// we can't allow plugins to depend on one another; we can't
+	// guarantee the order that plugins are loaded in.
+	foundStandardDir := slices.Contains(defaultDirectiveOrder, standardDir)
+	if !foundStandardDir {
+		panic("the 3rd argument '" + standardDir + "' must be a directive that exists in the standard distribution of Caddy")
+	}
+
+	// insert directive into proper position
+	newOrder := directiveOrder
+	for i, d := range newOrder {
+		if d != standardDir {
+			continue
+		}
+		switch position {
+		case Before:
+			newOrder = append(newOrder[:i], append([]string{dir}, newOrder[i:]...)...)
+		case After:
+			newOrder = append(newOrder[:i+1], append([]string{dir}, newOrder[i+1:]...)...)
+		case First, Last:
+		}
+		break
+	}
+	directiveOrder = newOrder
+}
+
 // RegisterGlobalOption registers a unique global option opt with
 // an associated unmarshaling (setup) function. When the global
 // option opt is encountered in a Caddyfile, setupFunc will be
@@ -172,6 +231,7 @@ func (h Helper) Caddyfiles() []string {
 	for file := range files {
 		filesSlice = append(filesSlice, file)
 	}
+	sort.Strings(filesSlice)
 	return filesSlice
 }
 
@@ -215,7 +275,8 @@ func (h Helper) ExtractMatcherSet() (caddy.ModuleMap, error) {
 
 // NewRoute returns config values relevant to creating a new HTTP route.
 func (h Helper) NewRoute(matcherSet caddy.ModuleMap,
-	handler caddyhttp.MiddlewareHandler) []ConfigValue {
+	handler caddyhttp.MiddlewareHandler,
+) []ConfigValue {
 	mod, err := caddy.GetModule(caddy.GetModuleID(handler))
 	if err != nil {
 		*h.warnings = append(*h.warnings, caddyconfig.Warning{
@@ -267,12 +328,6 @@ func (h Helper) GroupRoutes(vals []ConfigValue) {
 	}
 }
 
-// NewBindAddresses returns config values relevant to adding
-// listener bind addresses to the config.
-func (h Helper) NewBindAddresses(addrs []string) []ConfigValue {
-	return []ConfigValue{{Class: "bind", Value: addrs}}
-}
-
 // WithDispenser returns a new instance based on d. All others Helper
 // fields are copied, so typically maps are shared with this new instance.
 func (h Helper) WithDispenser(d *caddyfile.Dispenser) Helper {
@@ -289,7 +344,7 @@ func ParseSegmentAsSubroute(h Helper) (caddyhttp.MiddlewareHandler, error) {
 		return nil, err
 	}
 
-	return buildSubroute(allResults, h.groupCounter)
+	return buildSubroute(allResults, h.groupCounter, true)
 }
 
 // parseSegmentAsConfig parses the segment such that its subdirectives
@@ -313,9 +368,7 @@ func parseSegmentAsConfig(h Helper) ([]ConfigValue, error) {
 		// copy existing matcher definitions so we can augment
 		// new ones that are defined only in this scope
 		matcherDefs := make(map[string]caddy.ModuleMap, len(h.matcherDefs))
-		for key, val := range h.matcherDefs {
-			matcherDefs[key] = val
-		}
+		maps.Copy(matcherDefs, h.matcherDefs)
 
 		// find and extract any embedded matcher definitions in this scope
 		for i := 0; i < len(segments); i++ {
@@ -427,26 +480,33 @@ func sortRoutes(routes []ConfigValue) {
 			jPathLen = len(jPM[0])
 		}
 
-		// some directives involve setting values which can overwrite
-		// each other, so it makes most sense to reverse the order so
-		// that the lease specific matcher is first; everything else
-		// has most-specific matcher first
-		if iDir == "vars" {
+		sortByPath := func() bool {
 			// we can only confidently compare path lengths if both
 			// directives have a single path to match (issue #5037)
 			if iPathLen > 0 && jPathLen > 0 {
-				// sort least-specific (shortest) path first
-				return iPathLen < jPathLen
-			}
+				// trim the trailing wildcard if there is one
+				iPathTrimmed := strings.TrimSuffix(iPM[0], "*")
+				jPathTrimmed := strings.TrimSuffix(jPM[0], "*")
+
+				// if both paths are the same except for a trailing wildcard,
+				// sort by the shorter path first (which is more specific)
+				if iPathTrimmed == jPathTrimmed {
+					return iPathLen < jPathLen
+				}
+
+				// we use the trimmed length to compare the paths
+				// https://github.com/caddyserver/caddy/issues/7012#issuecomment-2870142195
+				// credit to https://github.com/Hellio404
+				// for sorts with many items, mixing matchers w/ and w/o wildcards will confuse the sort and result in incorrect orders
+				iPathLen = len(iPathTrimmed)
+				jPathLen = len(jPathTrimmed)
+
+				// if both paths have the same length, sort lexically
+				// https://github.com/caddyserver/caddy/pull/7015#issuecomment-2871993588
+				if iPathLen == jPathLen {
+					return iPathTrimmed < jPathTrimmed
+				}
 
-			// if both directives don't have a single path to compare,
-			// sort whichever one has no matcher first; if both have
-			// no matcher, sort equally (stable sort preserves order)
-			return len(iRoute.MatcherSetsRaw) == 0 && len(jRoute.MatcherSetsRaw) > 0
-		} else {
-			// we can only confidently compare path lengths if both
-			// directives have a single path to match (issue #5037)
-			if iPathLen > 0 && jPathLen > 0 {
 				// sort most-specific (longest) path first
 				return iPathLen > jPathLen
 			}
@@ -455,7 +515,18 @@ func sortRoutes(routes []ConfigValue) {
 			// sort whichever one has a matcher first; if both have
 			// a matcher, sort equally (stable sort preserves order)
 			return len(iRoute.MatcherSetsRaw) > 0 && len(jRoute.MatcherSetsRaw) == 0
+		}()
+
+		// some directives involve setting values which can overwrite
+		// each other, so it makes most sense to reverse the order so
+		// that the least-specific matcher is first, allowing the last
+		// matching one to win
+		if iDir == "vars" {
+			return !sortByPath
 		}
+
+		// everything else is most-specific matcher first
+		return sortByPath
 	})
 }
 
@@ -463,9 +534,9 @@ func sortRoutes(routes []ConfigValue) {
 // a "pile" of config values, keyed by class name,
 // as well as its parsed keys for convenience.
 type serverBlock struct {
-	block caddyfile.ServerBlock
-	pile  map[string][]ConfigValue // config values obtained from directives
-	keys  []Address
+	block      caddyfile.ServerBlock
+	pile       map[string][]ConfigValue // config values obtained from directives
+	parsedKeys []Address
 }
 
 // hostsFromKeys returns a list of all the non-empty hostnames found in
@@ -482,7 +553,7 @@ type serverBlock struct {
 func (sb serverBlock) hostsFromKeys(loggerMode bool) []string {
 	// ensure each entry in our list is unique
 	hostMap := make(map[string]struct{})
-	for _, addr := range sb.keys {
+	for _, addr := range sb.parsedKeys {
 		if addr.Host == "" {
 			if !loggerMode {
 				// server block contains a key like ":443", i.e. the host portion
@@ -514,7 +585,7 @@ func (sb serverBlock) hostsFromKeys(loggerMode bool) []string {
 func (sb serverBlock) hostsFromKeysNotHTTP(httpPort string) []string {
 	// ensure each entry in our list is unique
 	hostMap := make(map[string]struct{})
-	for _, addr := range sb.keys {
+	for _, addr := range sb.parsedKeys {
 		if addr.Host == "" {
 			continue
 		}
@@ -535,25 +606,29 @@ func (sb serverBlock) hostsFromKeysNotHTTP(httpPort string) []string {
 // hasHostCatchAllKey returns true if sb has a key that
 // omits a host portion, i.e. it "catches all" hosts.
 func (sb serverBlock) hasHostCatchAllKey() bool {
-	for _, addr := range sb.keys {
-		if addr.Host == "" {
-			return true
-		}
-	}
-	return false
+	return slices.ContainsFunc(sb.parsedKeys, func(addr Address) bool {
+		return addr.Host == ""
+	})
 }
 
 // isAllHTTP returns true if all sb keys explicitly specify
 // the http:// scheme
 func (sb serverBlock) isAllHTTP() bool {
-	for _, addr := range sb.keys {
-		if addr.Scheme != "http" {
-			return false
-		}
-	}
-	return true
+	return !slices.ContainsFunc(sb.parsedKeys, func(addr Address) bool {
+		return addr.Scheme != "http"
+	})
 }
 
+// Positional are the supported modes for ordering directives.
+type Positional string
+
+const (
+	Before Positional = "before"
+	After  Positional = "after"
+	First  Positional = "first"
+	Last   Positional = "last"
+)
+
 type (
 	// UnmarshalFunc is a function which can unmarshal Caddyfile
 	// tokens into zero or more config values using a Helper type.

caddyconfig/httpcaddyfile/directives_test.go

@@ -31,20 +31,23 @@ func TestHostsFromKeys(t *testing.T) {
 			[]Address{
 				{Original: ":2015", Port: "2015"},
 			},
-			[]string{}, []string{},
+			[]string{},
+			[]string{},
 		},
 		{
 			[]Address{
 				{Original: ":443", Port: "443"},
 			},
-			[]string{}, []string{},
+			[]string{},
+			[]string{},
 		},
 		{
 			[]Address{
 				{Original: "foo", Host: "foo"},
 				{Original: ":2015", Port: "2015"},
 			},
-			[]string{}, []string{"foo"},
+			[]string{},
+			[]string{"foo"},
 		},
 		{
 			[]Address{
@@ -75,7 +78,7 @@ func TestHostsFromKeys(t *testing.T) {
 			[]string{"example.com:2015"},
 		},
 	} {
-		sb := serverBlock{keys: tc.keys}
+		sb := serverBlock{parsedKeys: tc.keys}
 
 		// test in normal mode
 		actual := sb.hostsFromKeys(false)

caddyconfig/httpcaddyfile/options.go

@@ -15,32 +15,38 @@
 package httpcaddyfile
 
 import (
+	"slices"
 	"strconv"
 
+	"github.com/caddyserver/certmagic"
+	"github.com/libdns/libdns"
+	"github.com/mholt/acmez/v3/acme"
+
 	"github.com/caddyserver/caddy/v2"
 	"github.com/caddyserver/caddy/v2/caddyconfig"
 	"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
+	"github.com/caddyserver/caddy/v2/modules/caddyhttp"
 	"github.com/caddyserver/caddy/v2/modules/caddytls"
-	"github.com/caddyserver/certmagic"
-	"github.com/mholt/acmez/acme"
 )
 
 func init() {
 	RegisterGlobalOption("debug", parseOptTrue)
 	RegisterGlobalOption("http_port", parseOptHTTPPort)
 	RegisterGlobalOption("https_port", parseOptHTTPSPort)
-	RegisterGlobalOption("default_bind", parseOptStringList)
+	RegisterGlobalOption("default_bind", parseOptDefaultBind)
 	RegisterGlobalOption("grace_period", parseOptDuration)
 	RegisterGlobalOption("shutdown_delay", parseOptDuration)
 	RegisterGlobalOption("default_sni", parseOptSingleString)
+	RegisterGlobalOption("fallback_sni", parseOptSingleString)
 	RegisterGlobalOption("order", parseOptOrder)
 	RegisterGlobalOption("storage", parseOptStorage)
-	RegisterGlobalOption("storage_clean_interval", parseOptDuration)
+	RegisterGlobalOption("storage_check", parseStorageCheck)
+	RegisterGlobalOption("storage_clean_interval", parseStorageCleanInterval)
 	RegisterGlobalOption("renew_interval", parseOptDuration)
 	RegisterGlobalOption("ocsp_interval", parseOptDuration)
 	RegisterGlobalOption("acme_ca", parseOptSingleString)
 	RegisterGlobalOption("acme_ca_root", parseOptSingleString)
-	RegisterGlobalOption("acme_dns", parseOptACMEDNS)
+	RegisterGlobalOption("acme_dns", parseOptDNS)
 	RegisterGlobalOption("acme_eab", parseOptACMEEAB)
 	RegisterGlobalOption("cert_issuer", parseOptCertIssuer)
 	RegisterGlobalOption("skip_install_trust", parseOptTrue)
@@ -50,117 +56,119 @@ func init() {
 	RegisterGlobalOption("local_certs", parseOptTrue)
 	RegisterGlobalOption("key_type", parseOptSingleString)
 	RegisterGlobalOption("auto_https", parseOptAutoHTTPS)
+	RegisterGlobalOption("metrics", parseMetricsOptions)
 	RegisterGlobalOption("servers", parseServerOptions)
 	RegisterGlobalOption("ocsp_stapling", parseOCSPStaplingOptions)
+	RegisterGlobalOption("cert_lifetime", parseOptDuration)
 	RegisterGlobalOption("log", parseLogOptions)
 	RegisterGlobalOption("preferred_chains", parseOptPreferredChains)
+	RegisterGlobalOption("persist_config", parseOptPersistConfig)
+	RegisterGlobalOption("dns", parseOptDNS)
+	RegisterGlobalOption("ech", parseOptECH)
 }
 
 func parseOptTrue(d *caddyfile.Dispenser, _ any) (any, error) { return true, nil }
 
 func parseOptHTTPPort(d *caddyfile.Dispenser, _ any) (any, error) {
+	d.Next() // consume option name
 	var httpPort int
-	for d.Next() {
-		var httpPortStr string
-		if !d.AllArgs(&httpPortStr) {
-			return 0, d.ArgErr()
-		}
-		var err error
-		httpPort, err = strconv.Atoi(httpPortStr)
-		if err != nil {
-			return 0, d.Errf("converting port '%s' to integer value: %v", httpPortStr, err)
-		}
+	var httpPortStr string
+	if !d.AllArgs(&httpPortStr) {
+		return 0, d.ArgErr()
+	}
+	var err error
+	httpPort, err = strconv.Atoi(httpPortStr)
+	if err != nil {
+		return 0, d.Errf("converting port '%s' to integer value: %v", httpPortStr, err)
 	}
 	return httpPort, nil
 }
 
 func parseOptHTTPSPort(d *caddyfile.Dispenser, _ any) (any, error) {
+	d.Next() // consume option name
 	var httpsPort int
-	for d.Next() {
-		var httpsPortStr string
-		if !d.AllArgs(&httpsPortStr) {
-			return 0, d.ArgErr()
-		}
-		var err error
-		httpsPort, err = strconv.Atoi(httpsPortStr)
-		if err != nil {
-			return 0, d.Errf("converting port '%s' to integer value: %v", httpsPortStr, err)
-		}
+	var httpsPortStr string
+	if !d.AllArgs(&httpsPortStr) {
+		return 0, d.ArgErr()
+	}
+	var err error
+	httpsPort, err = strconv.Atoi(httpsPortStr)
+	if err != nil {
+		return 0, d.Errf("converting port '%s' to integer value: %v", httpsPortStr, err)
 	}
 	return httpsPort, nil
 }
 
 func parseOptOrder(d *caddyfile.Dispenser, _ any) (any, error) {
-	newOrder := directiveOrder
+	d.Next() // consume option name
 
-	for d.Next() {
-		// get directive name
-		if !d.Next() {
-			return nil, d.ArgErr()
-		}
-		dirName := d.Val()
-		if _, ok := registeredDirectives[dirName]; !ok {
-			return nil, d.Errf("%s is not a registered directive", dirName)
-		}
+	// get directive name
+	if !d.Next() {
+		return nil, d.ArgErr()
+	}
+	dirName := d.Val()
+	if _, ok := registeredDirectives[dirName]; !ok {
+		return nil, d.Errf("%s is not a registered directive", dirName)
+	}
 
-		// get positional token
-		if !d.Next() {
-			return nil, d.ArgErr()
-		}
-		pos := d.Val()
+	// get positional token
+	if !d.Next() {
+		return nil, d.ArgErr()
+	}
+	pos := Positional(d.Val())
 
-		// if directive exists, first remove it
-		for i, d := range newOrder {
-			if d == dirName {
-				newOrder = append(newOrder[:i], newOrder[i+1:]...)
-				break
-			}
-		}
+	// if directive already had an order, drop it
+	newOrder := slices.DeleteFunc(directiveOrder, func(d string) bool {
+		return d == dirName
+	})
 
-		// act on the positional
-		switch pos {
-		case "first":
-			newOrder = append([]string{dirName}, newOrder...)
-			if d.NextArg() {
-				return nil, d.ArgErr()
-			}
-			directiveOrder = newOrder
-			return newOrder, nil
-		case "last":
-			newOrder = append(newOrder, dirName)
-			if d.NextArg() {
-				return nil, d.ArgErr()
-			}
-			directiveOrder = newOrder
-			return newOrder, nil
-		case "before":
-		case "after":
-		default:
-			return nil, d.Errf("unknown positional '%s'", pos)
-		}
-
-		// get name of other directive
-		if !d.NextArg() {
-			return nil, d.ArgErr()
-		}
-		otherDir := d.Val()
+	// act on the positional; if it's First or Last, we're done right away
+	switch pos {
+	case First:
+		newOrder = append([]string{dirName}, newOrder...)
 		if d.NextArg() {
 			return nil, d.ArgErr()
 		}
+		directiveOrder = newOrder
+		return newOrder, nil
 
-		// insert directive into proper position
-		for i, d := range newOrder {
-			if d == otherDir {
-				if pos == "before" {
-					newOrder = append(newOrder[:i], append([]string{dirName}, newOrder[i:]...)...)
-				} else if pos == "after" {
-					newOrder = append(newOrder[:i+1], append([]string{dirName}, newOrder[i+1:]...)...)
-				}
-				break
-			}
+	case Last:
+		newOrder = append(newOrder, dirName)
+		if d.NextArg() {
+			return nil, d.ArgErr()
 		}
+		directiveOrder = newOrder
+		return newOrder, nil
+
+	// if it's Before or After, continue
+	case Before:
+	case After:
+
+	default:
+		return nil, d.Errf("unknown positional '%s'", pos)
 	}
 
+	// get name of other directive
+	if !d.NextArg() {
+		return nil, d.ArgErr()
+	}
+	otherDir := d.Val()
+	if d.NextArg() {
+		return nil, d.ArgErr()
+	}
+
+	// get the position of the target directive
+	targetIndex := slices.Index(newOrder, otherDir)
+	if targetIndex == -1 {
+		return nil, d.Errf("directive '%s' not found", otherDir)
+	}
+	// if we're inserting after, we need to increment the index to go after
+	if pos == After {
+		targetIndex++
+	}
+	// insert the directive into the new order
+	newOrder = slices.Insert(newOrder, targetIndex, dirName)
+
 	directiveOrder = newOrder
 
 	return newOrder, nil
@@ -185,6 +193,40 @@ func parseOptStorage(d *caddyfile.Dispenser, _ any) (any, error) {
 	return storage, nil
 }
 
+func parseStorageCheck(d *caddyfile.Dispenser, _ any) (any, error) {
+	d.Next() // consume option name
+	if !d.Next() {
+		return "", d.ArgErr()
+	}
+	val := d.Val()
+	if d.Next() {
+		return "", d.ArgErr()
+	}
+	if val != "off" {
+		return "", d.Errf("storage_check must be 'off'")
+	}
+	return val, nil
+}
+
+func parseStorageCleanInterval(d *caddyfile.Dispenser, _ any) (any, error) {
+	d.Next() // consume option name
+	if !d.Next() {
+		return "", d.ArgErr()
+	}
+	val := d.Val()
+	if d.Next() {
+		return "", d.ArgErr()
+	}
+	if val == "off" {
+		return false, nil
+	}
+	dur, err := caddy.ParseDuration(d.Val())
+	if err != nil {
+		return nil, d.Errf("failed to parse storage_clean_interval, must be a duration or 'off' %w", err)
+	}
+	return caddy.Duration(dur), nil
+}
+
 func parseOptDuration(d *caddyfile.Dispenser, _ any) (any, error) {
 	if !d.Next() { // consume option name
 		return nil, d.ArgErr()
@@ -199,78 +241,60 @@ func parseOptDuration(d *caddyfile.Dispenser, _ any) (any, error) {
 	return caddy.Duration(dur), nil
 }
 
-func parseOptACMEDNS(d *caddyfile.Dispenser, _ any) (any, error) {
-	if !d.Next() { // consume option name
-		return nil, d.ArgErr()
-	}
-	if !d.Next() { // get DNS module name
-		return nil, d.ArgErr()
-	}
-	modID := "dns.providers." + d.Val()
-	unm, err := caddyfile.UnmarshalModule(d, modID)
-	if err != nil {
-		return nil, err
-	}
-	prov, ok := unm.(certmagic.ACMEDNSProvider)
-	if !ok {
-		return nil, d.Errf("module %s (%T) is not a certmagic.ACMEDNSProvider", modID, unm)
-	}
-	return prov, nil
-}
-
 func parseOptACMEEAB(d *caddyfile.Dispenser, _ any) (any, error) {
 	eab := new(acme.EAB)
-	for d.Next() {
-		if d.NextArg() {
-			return nil, d.ArgErr()
-		}
-		for nesting := d.Nesting(); d.NextBlock(nesting); {
-			switch d.Val() {
-			case "key_id":
-				if !d.NextArg() {
-					return nil, d.ArgErr()
-				}
-				eab.KeyID = d.Val()
-
-			case "mac_key":
-				if !d.NextArg() {
-					return nil, d.ArgErr()
-				}
-				eab.MACKey = d.Val()
-
-			default:
-				return nil, d.Errf("unrecognized parameter '%s'", d.Val())
+	d.Next() // consume option name
+	if d.NextArg() {
+		return nil, d.ArgErr()
+	}
+	for d.NextBlock(0) {
+		switch d.Val() {
+		case "key_id":
+			if !d.NextArg() {
+				return nil, d.ArgErr()
 			}
+			eab.KeyID = d.Val()
+
+		case "mac_key":
+			if !d.NextArg() {
+				return nil, d.ArgErr()
+			}
+			eab.MACKey = d.Val()
+
+		default:
+			return nil, d.Errf("unrecognized parameter '%s'", d.Val())
 		}
 	}
 	return eab, nil
 }
 
 func parseOptCertIssuer(d *caddyfile.Dispenser, existing any) (any, error) {
+	d.Next() // consume option name
+
 	var issuers []certmagic.Issuer
 	if existing != nil {
 		issuers = existing.([]certmagic.Issuer)
 	}
-	for d.Next() { // consume option name
-		if !d.Next() { // get issuer module name
-			return nil, d.ArgErr()
-		}
-		modID := "tls.issuance." + d.Val()
-		unm, err := caddyfile.UnmarshalModule(d, modID)
-		if err != nil {
-			return nil, err
-		}
-		iss, ok := unm.(certmagic.Issuer)
-		if !ok {
-			return nil, d.Errf("module %s (%T) is not a certmagic.Issuer", modID, unm)
-		}
-		issuers = append(issuers, iss)
+
+	// get issuer module name
+	if !d.Next() {
+		return nil, d.ArgErr()
 	}
+	modID := "tls.issuance." + d.Val()
+	unm, err := caddyfile.UnmarshalModule(d, modID)
+	if err != nil {
+		return nil, err
+	}
+	iss, ok := unm.(certmagic.Issuer)
+	if !ok {
+		return nil, d.Errf("module %s (%T) is not a certmagic.Issuer", modID, unm)
+	}
+	issuers = append(issuers, iss)
 	return issuers, nil
 }
 
 func parseOptSingleString(d *caddyfile.Dispenser, _ any) (any, error) {
-	d.Next() // consume parameter name
+	d.Next() // consume option name
 	if !d.Next() {
 		return "", d.ArgErr()
 	}
@@ -281,43 +305,62 @@ func parseOptSingleString(d *caddyfile.Dispenser, _ any) (any, error) {
 	return val, nil
 }
 
-func parseOptStringList(d *caddyfile.Dispenser, _ any) (any, error) {
-	d.Next() // consume parameter name
-	val := d.RemainingArgs()
-	if len(val) == 0 {
-		return "", d.ArgErr()
+func parseOptDefaultBind(d *caddyfile.Dispenser, _ any) (any, error) {
+	d.Next() // consume option name
+
+	var addresses, protocols []string
+	addresses = d.RemainingArgs()
+
+	if len(addresses) == 0 {
+		addresses = append(addresses, "")
 	}
-	return val, nil
+
+	for d.NextBlock(0) {
+		switch d.Val() {
+		case "protocols":
+			protocols = d.RemainingArgs()
+			if len(protocols) == 0 {
+				return nil, d.Errf("protocols requires one or more arguments")
+			}
+		default:
+			return nil, d.Errf("unknown subdirective: %s", d.Val())
+		}
+	}
+
+	return []ConfigValue{{Class: "bind", Value: addressesWithProtocols{
+		addresses: addresses,
+		protocols: protocols,
+	}}}, nil
 }
 
 func parseOptAdmin(d *caddyfile.Dispenser, _ any) (any, error) {
+	d.Next() // consume option name
+
 	adminCfg := new(caddy.AdminConfig)
-	for d.Next() {
-		if d.NextArg() {
-			listenAddress := d.Val()
-			if listenAddress == "off" {
-				adminCfg.Disabled = true
-				if d.Next() { // Do not accept any remaining options including block
-					return nil, d.Err("No more option is allowed after turning off admin config")
-				}
-			} else {
-				adminCfg.Listen = listenAddress
-				if d.NextArg() { // At most 1 arg is allowed
-					return nil, d.ArgErr()
-				}
+	if d.NextArg() {
+		listenAddress := d.Val()
+		if listenAddress == "off" {
+			adminCfg.Disabled = true
+			if d.Next() { // Do not accept any remaining options including block
+				return nil, d.Err("No more option is allowed after turning off admin config")
+			}
+		} else {
+			adminCfg.Listen = listenAddress
+			if d.NextArg() { // At most 1 arg is allowed
+				return nil, d.ArgErr()
 			}
 		}
-		for nesting := d.Nesting(); d.NextBlock(nesting); {
-			switch d.Val() {
-			case "enforce_origin":
-				adminCfg.EnforceOrigin = true
+	}
+	for d.NextBlock(0) {
+		switch d.Val() {
+		case "enforce_origin":
+			adminCfg.EnforceOrigin = true
 
-			case "origins":
-				adminCfg.Origins = d.RemainingArgs()
+		case "origins":
+			adminCfg.Origins = d.RemainingArgs()
 
-			default:
-				return nil, d.Errf("unrecognized parameter '%s'", d.Val())
-			}
+		default:
+			return nil, d.Errf("unrecognized parameter '%s'", d.Val())
 		}
 	}
 	if adminCfg.Listen == "" && !adminCfg.Disabled {
@@ -327,57 +370,58 @@ func parseOptAdmin(d *caddyfile.Dispenser, _ any) (any, error) {
 }
 
 func parseOptOnDemand(d *caddyfile.Dispenser, _ any) (any, error) {
+	d.Next() // consume option name
+	if d.NextArg() {
+		return nil, d.ArgErr()
+	}
+
 	var ond *caddytls.OnDemandConfig
-	for d.Next() {
-		if d.NextArg() {
-			return nil, d.ArgErr()
-		}
-		for nesting := d.Nesting(); d.NextBlock(nesting); {
-			switch d.Val() {
-			case "ask":
-				if !d.NextArg() {
-					return nil, d.ArgErr()
-				}
-				if ond == nil {
-					ond = new(caddytls.OnDemandConfig)
-				}
-				ond.Ask = d.Val()
 
-			case "interval":
-				if !d.NextArg() {
-					return nil, d.ArgErr()
-				}
-				dur, err := caddy.ParseDuration(d.Val())
-				if err != nil {
-					return nil, err
-				}
-				if ond == nil {
-					ond = new(caddytls.OnDemandConfig)
-				}
-				if ond.RateLimit == nil {
-					ond.RateLimit = new(caddytls.RateLimit)
-				}
-				ond.RateLimit.Interval = caddy.Duration(dur)
-
-			case "burst":
-				if !d.NextArg() {
-					return nil, d.ArgErr()
-				}
-				burst, err := strconv.Atoi(d.Val())
-				if err != nil {
-					return nil, err
-				}
-				if ond == nil {
-					ond = new(caddytls.OnDemandConfig)
-				}
-				if ond.RateLimit == nil {
-					ond.RateLimit = new(caddytls.RateLimit)
-				}
-				ond.RateLimit.Burst = burst
-
-			default:
-				return nil, d.Errf("unrecognized parameter '%s'", d.Val())
+	for nesting := d.Nesting(); d.NextBlock(nesting); {
+		switch d.Val() {
+		case "ask":
+			if !d.NextArg() {
+				return nil, d.ArgErr()
 			}
+			if ond == nil {
+				ond = new(caddytls.OnDemandConfig)
+			}
+			if ond.PermissionRaw != nil {
+				return nil, d.Err("on-demand TLS permission module (or 'ask') already specified")
+			}
+			perm := caddytls.PermissionByHTTP{Endpoint: d.Val()}
+			ond.PermissionRaw = caddyconfig.JSONModuleObject(perm, "module", "http", nil)
+
+		case "permission":
+			if !d.NextArg() {
+				return nil, d.ArgErr()
+			}
+			if ond == nil {
+				ond = new(caddytls.OnDemandConfig)
+			}
+			if ond.PermissionRaw != nil {
+				return nil, d.Err("on-demand TLS permission module (or 'ask') already specified")
+			}
+			modName := d.Val()
+			modID := "tls.permission." + modName
+			unm, err := caddyfile.UnmarshalModule(d, modID)
+			if err != nil {
+				return nil, err
+			}
+			perm, ok := unm.(caddytls.OnDemandPermission)
+			if !ok {
+				return nil, d.Errf("module %s (%T) is not an on-demand TLS permission module", modID, unm)
+			}
+			ond.PermissionRaw = caddyconfig.JSONModuleObject(perm, "module", modName, nil)
+
+		case "interval":
+			return nil, d.Errf("the on_demand_tls 'interval' option is no longer supported, remove it from your config")
+
+		case "burst":
+			return nil, d.Errf("the on_demand_tls 'burst' option is no longer supported, remove it from your config")
+
+		default:
+			return nil, d.Errf("unrecognized parameter '%s'", d.Val())
 		}
 	}
 	if ond == nil {
@@ -386,8 +430,8 @@ func parseOptOnDemand(d *caddyfile.Dispenser, _ any) (any, error) {
 	return ond, nil
 }
 
-func parseOptAutoHTTPS(d *caddyfile.Dispenser, _ any) (any, error) {
-	d.Next() // consume parameter name
+func parseOptPersistConfig(d *caddyfile.Dispenser, _ any) (any, error) {
+	d.Next() // consume option name
 	if !d.Next() {
 		return "", d.ArgErr()
 	}
@@ -395,12 +439,52 @@ func parseOptAutoHTTPS(d *caddyfile.Dispenser, _ any) (any, error) {
 	if d.Next() {
 		return "", d.ArgErr()
 	}
-	if val != "off" && val != "disable_redirects" && val != "disable_certs" && val != "ignore_loaded_certs" {
-		return "", d.Errf("auto_https must be one of 'off', 'disable_redirects', 'disable_certs', or 'ignore_loaded_certs'")
+	if val != "off" {
+		return "", d.Errf("persist_config must be 'off'")
 	}
 	return val, nil
 }
 
+func parseOptAutoHTTPS(d *caddyfile.Dispenser, _ any) (any, error) {
+	d.Next() // consume option name
+	val := d.RemainingArgs()
+	if len(val) == 0 {
+		return "", d.ArgErr()
+	}
+	for _, v := range val {
+		switch v {
+		case "off":
+		case "disable_redirects":
+		case "disable_certs":
+		case "ignore_loaded_certs":
+		case "prefer_wildcard":
+			break
+
+		default:
+			return "", d.Errf("auto_https must be one of 'off', 'disable_redirects', 'disable_certs', 'ignore_loaded_certs', or 'prefer_wildcard'")
+		}
+	}
+	return val, nil
+}
+
+func unmarshalCaddyfileMetricsOptions(d *caddyfile.Dispenser) (any, error) {
+	d.Next() // consume option name
+	metrics := new(caddyhttp.Metrics)
+	for d.NextBlock(0) {
+		switch d.Val() {
+		case "per_host":
+			metrics.PerHost = true
+		default:
+			return nil, d.Errf("unrecognized servers option '%s'", d.Val())
+		}
+	}
+	return metrics, nil
+}
+
+func parseMetricsOptions(d *caddyfile.Dispenser, _ any) (any, error) {
+	return unmarshalCaddyfileMetricsOptions(d)
+}
+
 func parseServerOptions(d *caddyfile.Dispenser, _ any) (any, error) {
 	return unmarshalCaddyfileServerOptions(d)
 }
@@ -470,3 +554,74 @@ func parseOptPreferredChains(d *caddyfile.Dispenser, _ any) (any, error) {
 	d.Next()
 	return caddytls.ParseCaddyfilePreferredChainsOptions(d)
 }
+
+func parseOptDNS(d *caddyfile.Dispenser, _ any) (any, error) {
+	d.Next() // consume option name
+	optName := d.Val()
+
+	// get DNS module name
+	if !d.Next() {
+		// this is allowed if this is the "acme_dns" option since it may refer to the globally-configured "dns" option's value
+		if optName == "acme_dns" {
+			return nil, nil
+		}
+		return nil, d.ArgErr()
+	}
+	modID := "dns.providers." + d.Val()
+	unm, err := caddyfile.UnmarshalModule(d, modID)
+	if err != nil {
+		return nil, err
+	}
+	switch unm.(type) {
+	case libdns.RecordGetter,
+		libdns.RecordSetter,
+		libdns.RecordAppender,
+		libdns.RecordDeleter:
+	default:
+		return nil, d.Errf("module %s (%T) is not a libdns provider", modID, unm)
+	}
+	return unm, nil
+}
+
+func parseOptECH(d *caddyfile.Dispenser, _ any) (any, error) {
+	d.Next() // consume option name
+
+	ech := new(caddytls.ECH)
+
+	publicNames := d.RemainingArgs()
+	for _, publicName := range publicNames {
+		ech.Configs = append(ech.Configs, caddytls.ECHConfiguration{
+			PublicName: publicName,
+		})
+	}
+	if len(ech.Configs) == 0 {
+		return nil, d.ArgErr()
+	}
+
+	for nesting := d.Nesting(); d.NextBlock(nesting); {
+		switch d.Val() {
+		case "dns":
+			if !d.Next() {
+				return nil, d.ArgErr()
+			}
+			providerName := d.Val()
+			modID := "dns.providers." + providerName
+			unm, err := caddyfile.UnmarshalModule(d, modID)
+			if err != nil {
+				return nil, err
+			}
+			ech.Publication = append(ech.Publication, &caddytls.ECHPublication{
+				Configs: publicNames,
+				PublishersRaw: caddy.ModuleMap{
+					"dns": caddyconfig.JSON(caddytls.ECHDNSPublisher{
+						ProviderRaw: caddyconfig.JSONModuleObject(unm, "name", providerName, nil),
+					}, nil),
+				},
+			})
+		default:
+			return nil, d.Errf("ech: unrecognized subdirective '%s'", d.Val())
+		}
+	}
+
+	return ech, nil
+}

caddyconfig/httpcaddyfile/pkiapp.go

@@ -15,6 +15,7 @@
 package httpcaddyfile
 
 import (
+	"github.com/caddyserver/caddy/v2"
 	"github.com/caddyserver/caddy/v2/caddyconfig"
 	"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
 	"github.com/caddyserver/caddy/v2/modules/caddypki"
@@ -26,134 +27,145 @@ func init() {
 
 // parsePKIApp parses the global log option. Syntax:
 //
-//     pki {
-//         ca [<id>] {
-//             name            <name>
-//             root_cn         <name>
-//             intermediate_cn <name>
-//             root {
-//                 cert   <path>
-//                 key    <path>
-//                 format <format>
-//             }
-//             intermediate {
-//                 cert   <path>
-//                 key    <path>
-//                 format <format>
-//             }
-//         }
-//     }
+//	pki {
+//	    ca [<id>] {
+//	        name                  <name>
+//	        root_cn               <name>
+//	        intermediate_cn       <name>
+//	        intermediate_lifetime <duration>
+//	        root {
+//	            cert   <path>
+//	            key    <path>
+//	            format <format>
+//	        }
+//	        intermediate {
+//	            cert   <path>
+//	            key    <path>
+//	            format <format>
+//	        }
+//	    }
+//	}
 //
 // When the CA ID is unspecified, 'local' is assumed.
 func parsePKIApp(d *caddyfile.Dispenser, existingVal any) (any, error) {
-	pki := &caddypki.PKI{CAs: make(map[string]*caddypki.CA)}
+	d.Next() // consume app name
 
-	for d.Next() {
-		for nesting := d.Nesting(); d.NextBlock(nesting); {
-			switch d.Val() {
-			case "ca":
-				pkiCa := new(caddypki.CA)
+	pki := &caddypki.PKI{
+		CAs: make(map[string]*caddypki.CA),
+	}
+	for d.NextBlock(0) {
+		switch d.Val() {
+		case "ca":
+			pkiCa := new(caddypki.CA)
+			if d.NextArg() {
+				pkiCa.ID = d.Val()
 				if d.NextArg() {
-					pkiCa.ID = d.Val()
-					if d.NextArg() {
+					return nil, d.ArgErr()
+				}
+			}
+			if pkiCa.ID == "" {
+				pkiCa.ID = caddypki.DefaultCAID
+			}
+
+			for nesting := d.Nesting(); d.NextBlock(nesting); {
+				switch d.Val() {
+				case "name":
+					if !d.NextArg() {
 						return nil, d.ArgErr()
 					}
-				}
-				if pkiCa.ID == "" {
-					pkiCa.ID = caddypki.DefaultCAID
-				}
+					pkiCa.Name = d.Val()
 
-				for nesting := d.Nesting(); d.NextBlock(nesting); {
-					switch d.Val() {
-					case "name":
-						if !d.NextArg() {
-							return nil, d.ArgErr()
-						}
-						pkiCa.Name = d.Val()
-
-					case "root_cn":
-						if !d.NextArg() {
-							return nil, d.ArgErr()
-						}
-						pkiCa.RootCommonName = d.Val()
-
-					case "intermediate_cn":
-						if !d.NextArg() {
-							return nil, d.ArgErr()
-						}
-						pkiCa.IntermediateCommonName = d.Val()
-
-					case "root":
-						if pkiCa.Root == nil {
-							pkiCa.Root = new(caddypki.KeyPair)
-						}
-						for nesting := d.Nesting(); d.NextBlock(nesting); {
-							switch d.Val() {
-							case "cert":
-								if !d.NextArg() {
-									return nil, d.ArgErr()
-								}
-								pkiCa.Root.Certificate = d.Val()
-
-							case "key":
-								if !d.NextArg() {
-									return nil, d.ArgErr()
-								}
-								pkiCa.Root.PrivateKey = d.Val()
-
-							case "format":
-								if !d.NextArg() {
-									return nil, d.ArgErr()
-								}
-								pkiCa.Root.Format = d.Val()
-
-							default:
-								return nil, d.Errf("unrecognized pki ca root option '%s'", d.Val())
-							}
-						}
-
-					case "intermediate":
-						if pkiCa.Intermediate == nil {
-							pkiCa.Intermediate = new(caddypki.KeyPair)
-						}
-						for nesting := d.Nesting(); d.NextBlock(nesting); {
-							switch d.Val() {
-							case "cert":
-								if !d.NextArg() {
-									return nil, d.ArgErr()
-								}
-								pkiCa.Intermediate.Certificate = d.Val()
-
-							case "key":
-								if !d.NextArg() {
-									return nil, d.ArgErr()
-								}
-								pkiCa.Intermediate.PrivateKey = d.Val()
-
-							case "format":
-								if !d.NextArg() {
-									return nil, d.ArgErr()
-								}
-								pkiCa.Intermediate.Format = d.Val()
-
-							default:
-								return nil, d.Errf("unrecognized pki ca intermediate option '%s'", d.Val())
-							}
-						}
-
-					default:
-						return nil, d.Errf("unrecognized pki ca option '%s'", d.Val())
+				case "root_cn":
+					if !d.NextArg() {
+						return nil, d.ArgErr()
 					}
+					pkiCa.RootCommonName = d.Val()
+
+				case "intermediate_cn":
+					if !d.NextArg() {
+						return nil, d.ArgErr()
+					}
+					pkiCa.IntermediateCommonName = d.Val()
+
+				case "intermediate_lifetime":
+					if !d.NextArg() {
+						return nil, d.ArgErr()
+					}
+					dur, err := caddy.ParseDuration(d.Val())
+					if err != nil {
+						return nil, err
+					}
+					pkiCa.IntermediateLifetime = caddy.Duration(dur)
+
+				case "root":
+					if pkiCa.Root == nil {
+						pkiCa.Root = new(caddypki.KeyPair)
+					}
+					for nesting := d.Nesting(); d.NextBlock(nesting); {
+						switch d.Val() {
+						case "cert":
+							if !d.NextArg() {
+								return nil, d.ArgErr()
+							}
+							pkiCa.Root.Certificate = d.Val()
+
+						case "key":
+							if !d.NextArg() {
+								return nil, d.ArgErr()
+							}
+							pkiCa.Root.PrivateKey = d.Val()
+
+						case "format":
+							if !d.NextArg() {
+								return nil, d.ArgErr()
+							}
+							pkiCa.Root.Format = d.Val()
+
+						default:
+							return nil, d.Errf("unrecognized pki ca root option '%s'", d.Val())
+						}
+					}
+
+				case "intermediate":
+					if pkiCa.Intermediate == nil {
+						pkiCa.Intermediate = new(caddypki.KeyPair)
+					}
+					for nesting := d.Nesting(); d.NextBlock(nesting); {
+						switch d.Val() {
+						case "cert":
+							if !d.NextArg() {
+								return nil, d.ArgErr()
+							}
+							pkiCa.Intermediate.Certificate = d.Val()
+
+						case "key":
+							if !d.NextArg() {
+								return nil, d.ArgErr()
+							}
+							pkiCa.Intermediate.PrivateKey = d.Val()
+
+						case "format":
+							if !d.NextArg() {
+								return nil, d.ArgErr()
+							}
+							pkiCa.Intermediate.Format = d.Val()
+
+						default:
+							return nil, d.Errf("unrecognized pki ca intermediate option '%s'", d.Val())
+						}
+					}
+
+				default:
+					return nil, d.Errf("unrecognized pki ca option '%s'", d.Val())
 				}
-
-				pki.CAs[pkiCa.ID] = pkiCa
-
-			default:
-				return nil, d.Errf("unrecognized pki option '%s'", d.Val())
 			}
+
+			pki.CAs[pkiCa.ID] = pkiCa
+
+		default:
+			return nil, d.Errf("unrecognized pki option '%s'", d.Val())
 		}
 	}
-
 	return pki, nil
 }
 
@@ -162,7 +174,6 @@ func (st ServerType) buildPKIApp(
 	options map[string]any,
 	warnings []caddyconfig.Warning,
 ) (*caddypki.PKI, []caddyconfig.Warning, error) {
-
 	skipInstallTrust := false
 	if _, ok := options["skip_install_trust"]; ok {
 		skipInstallTrust = true

caddyconfig/httpcaddyfile/serveroptions.go

@@ -17,12 +17,14 @@ package httpcaddyfile
 import (
 	"encoding/json"
 	"fmt"
+	"slices"
+
+	"github.com/dustin/go-humanize"
 
 	"github.com/caddyserver/caddy/v2"
 	"github.com/caddyserver/caddy/v2/caddyconfig"
 	"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
 	"github.com/caddyserver/caddy/v2/modules/caddyhttp"
-	"github.com/dustin/go-humanize"
 )
 
 // serverOptions collects server config overrides parsed from Caddyfile global options
@@ -33,6 +35,7 @@ type serverOptions struct {
 	ListenerAddress string
 
 	// These will all map 1:1 to the caddyhttp.Server struct
+	Name                 string
 	ListenerWrappersRaw  []json.RawMessage
 	ReadTimeout          caddy.Duration
 	ReadHeaderTimeout    caddy.Duration
@@ -40,188 +43,222 @@ type serverOptions struct {
 	IdleTimeout          caddy.Duration
 	KeepAliveInterval    caddy.Duration
 	MaxHeaderBytes       int
+	EnableFullDuplex     bool
 	Protocols            []string
 	StrictSNIHost        *bool
+	TrustedProxiesRaw    json.RawMessage
+	TrustedProxiesStrict int
+	ClientIPHeaders      []string
 	ShouldLogCredentials bool
 	Metrics              *caddyhttp.Metrics
+	Trace                bool // TODO: EXPERIMENTAL
 }
 
 func unmarshalCaddyfileServerOptions(d *caddyfile.Dispenser) (any, error) {
+	d.Next() // consume option name
+
 	serverOpts := serverOptions{}
-	for d.Next() {
+	if d.NextArg() {
+		serverOpts.ListenerAddress = d.Val()
 		if d.NextArg() {
-			serverOpts.ListenerAddress = d.Val()
+			return nil, d.ArgErr()
+		}
+	}
+	for d.NextBlock(0) {
+		switch d.Val() {
+		case "name":
+			if serverOpts.ListenerAddress == "" {
+				return nil, d.Errf("cannot set a name for a server without a listener address")
+			}
+			if !d.NextArg() {
+				return nil, d.ArgErr()
+			}
+			serverOpts.Name = d.Val()
+
+		case "listener_wrappers":
+			for nesting := d.Nesting(); d.NextBlock(nesting); {
+				modID := "caddy.listeners." + d.Val()
+				unm, err := caddyfile.UnmarshalModule(d, modID)
+				if err != nil {
+					return nil, err
+				}
+				listenerWrapper, ok := unm.(caddy.ListenerWrapper)
+				if !ok {
+					return nil, fmt.Errorf("module %s (%T) is not a listener wrapper", modID, unm)
+				}
+				jsonListenerWrapper := caddyconfig.JSONModuleObject(
+					listenerWrapper,
+					"wrapper",
+					listenerWrapper.(caddy.Module).CaddyModule().ID.Name(),
+					nil,
+				)
+				serverOpts.ListenerWrappersRaw = append(serverOpts.ListenerWrappersRaw, jsonListenerWrapper)
+			}
+
+		case "timeouts":
+			for nesting := d.Nesting(); d.NextBlock(nesting); {
+				switch d.Val() {
+				case "read_body":
+					if !d.NextArg() {
+						return nil, d.ArgErr()
+					}
+					dur, err := caddy.ParseDuration(d.Val())
+					if err != nil {
+						return nil, d.Errf("parsing read_body timeout duration: %v", err)
+					}
+					serverOpts.ReadTimeout = caddy.Duration(dur)
+
+				case "read_header":
+					if !d.NextArg() {
+						return nil, d.ArgErr()
+					}
+					dur, err := caddy.ParseDuration(d.Val())
+					if err != nil {
+						return nil, d.Errf("parsing read_header timeout duration: %v", err)
+					}
+					serverOpts.ReadHeaderTimeout = caddy.Duration(dur)
+
+				case "write":
+					if !d.NextArg() {
+						return nil, d.ArgErr()
+					}
+					dur, err := caddy.ParseDuration(d.Val())
+					if err != nil {
+						return nil, d.Errf("parsing write timeout duration: %v", err)
+					}
+					serverOpts.WriteTimeout = caddy.Duration(dur)
+
+				case "idle":
+					if !d.NextArg() {
+						return nil, d.ArgErr()
+					}
+					dur, err := caddy.ParseDuration(d.Val())
+					if err != nil {
+						return nil, d.Errf("parsing idle timeout duration: %v", err)
+					}
+					serverOpts.IdleTimeout = caddy.Duration(dur)
+
+				default:
+					return nil, d.Errf("unrecognized timeouts option '%s'", d.Val())
+				}
+			}
+		case "keepalive_interval":
+			if !d.NextArg() {
+				return nil, d.ArgErr()
+			}
+			dur, err := caddy.ParseDuration(d.Val())
+			if err != nil {
+				return nil, d.Errf("parsing keepalive interval duration: %v", err)
+			}
+			serverOpts.KeepAliveInterval = caddy.Duration(dur)
+
+		case "max_header_size":
+			var sizeStr string
+			if !d.AllArgs(&sizeStr) {
+				return nil, d.ArgErr()
+			}
+			size, err := humanize.ParseBytes(sizeStr)
+			if err != nil {
+				return nil, d.Errf("parsing max_header_size: %v", err)
+			}
+			serverOpts.MaxHeaderBytes = int(size)
+
+		case "enable_full_duplex":
 			if d.NextArg() {
 				return nil, d.ArgErr()
 			}
-		}
-		for nesting := d.Nesting(); d.NextBlock(nesting); {
-			switch d.Val() {
-			case "listener_wrappers":
-				for nesting := d.Nesting(); d.NextBlock(nesting); {
-					modID := "caddy.listeners." + d.Val()
-					unm, err := caddyfile.UnmarshalModule(d, modID)
-					if err != nil {
-						return nil, err
-					}
-					listenerWrapper, ok := unm.(caddy.ListenerWrapper)
-					if !ok {
-						return nil, fmt.Errorf("module %s (%T) is not a listener wrapper", modID, unm)
-					}
-					jsonListenerWrapper := caddyconfig.JSONModuleObject(
-						listenerWrapper,
-						"wrapper",
-						listenerWrapper.(caddy.Module).CaddyModule().ID.Name(),
-						nil,
-					)
-					serverOpts.ListenerWrappersRaw = append(serverOpts.ListenerWrappersRaw, jsonListenerWrapper)
-				}
+			serverOpts.EnableFullDuplex = true
 
-			case "timeouts":
-				for nesting := d.Nesting(); d.NextBlock(nesting); {
-					switch d.Val() {
-					case "read_body":
-						if !d.NextArg() {
-							return nil, d.ArgErr()
-						}
-						dur, err := caddy.ParseDuration(d.Val())
-						if err != nil {
-							return nil, d.Errf("parsing read_body timeout duration: %v", err)
-						}
-						serverOpts.ReadTimeout = caddy.Duration(dur)
-
-					case "read_header":
-						if !d.NextArg() {
-							return nil, d.ArgErr()
-						}
-						dur, err := caddy.ParseDuration(d.Val())
-						if err != nil {
-							return nil, d.Errf("parsing read_header timeout duration: %v", err)
-						}
-						serverOpts.ReadHeaderTimeout = caddy.Duration(dur)
-
-					case "write":
-						if !d.NextArg() {
-							return nil, d.ArgErr()
-						}
-						dur, err := caddy.ParseDuration(d.Val())
-						if err != nil {
-							return nil, d.Errf("parsing write timeout duration: %v", err)
-						}
-						serverOpts.WriteTimeout = caddy.Duration(dur)
-
-					case "idle":
-						if !d.NextArg() {
-							return nil, d.ArgErr()
-						}
-						dur, err := caddy.ParseDuration(d.Val())
-						if err != nil {
-							return nil, d.Errf("parsing idle timeout duration: %v", err)
-						}
-						serverOpts.IdleTimeout = caddy.Duration(dur)
-
-					default:
-						return nil, d.Errf("unrecognized timeouts option '%s'", d.Val())
-					}
-				}
-			case "keepalive_interval":
-				if !d.NextArg() {
-					return nil, d.ArgErr()
-				}
-				dur, err := caddy.ParseDuration(d.Val())
-				if err != nil {
-					return nil, d.Errf("parsing keepalive interval duration: %v", err)
-				}
-				serverOpts.KeepAliveInterval = caddy.Duration(dur)
-
-			case "max_header_size":
-				var sizeStr string
-				if !d.AllArgs(&sizeStr) {
-					return nil, d.ArgErr()
-				}
-				size, err := humanize.ParseBytes(sizeStr)
-				if err != nil {
-					return nil, d.Errf("parsing max_header_size: %v", err)
-				}
-				serverOpts.MaxHeaderBytes = int(size)
-
-			case "log_credentials":
-				if d.NextArg() {
-					return nil, d.ArgErr()
-				}
-				serverOpts.ShouldLogCredentials = true
-
-			case "protocols":
-				protos := d.RemainingArgs()
-				for _, proto := range protos {
-					if proto != "h1" && proto != "h2" && proto != "h2c" && proto != "h3" {
-						return nil, d.Errf("unknown protocol '%s': expected h1, h2, h2c, or h3", proto)
-					}
-					if sliceContains(serverOpts.Protocols, proto) {
-						return nil, d.Errf("protocol %s specified more than once", proto)
-					}
-					serverOpts.Protocols = append(serverOpts.Protocols, proto)
-				}
-				if nesting := d.Nesting(); d.NextBlock(nesting) {
-					return nil, d.ArgErr()
-				}
-
-			case "strict_sni_host":
-				if d.NextArg() && d.Val() != "insecure_off" && d.Val() != "on" {
-					return nil, d.Errf("strict_sni_host only supports 'on' or 'insecure_off', got '%s'", d.Val())
-				}
-				boolVal := true
-				if d.Val() == "insecure_off" {
-					boolVal = false
-				}
-				serverOpts.StrictSNIHost = &boolVal
-
-			case "metrics":
-				if d.NextArg() {
-					return nil, d.ArgErr()
-				}
-				if d.NextBlock(0) {
-					return nil, d.ArgErr()
-				}
-				serverOpts.Metrics = new(caddyhttp.Metrics)
-
-			// TODO: DEPRECATED. (August 2022)
-			case "protocol":
-				caddy.Log().Named("caddyfile").Warn("DEPRECATED: protocol sub-option will be removed soon")
-
-				for nesting := d.Nesting(); d.NextBlock(nesting); {
-					switch d.Val() {
-					case "allow_h2c":
-						caddy.Log().Named("caddyfile").Warn("DEPRECATED: allow_h2c will be removed soon; use protocols option instead")
-
-						if d.NextArg() {
-							return nil, d.ArgErr()
-						}
-						if sliceContains(serverOpts.Protocols, "h2c") {
-							return nil, d.Errf("protocol h2c already specified")
-						}
-						serverOpts.Protocols = append(serverOpts.Protocols, "h2c")
-
-					case "strict_sni_host":
-						caddy.Log().Named("caddyfile").Warn("DEPRECATED: protocol > strict_sni_host in this position will be removed soon; move up to the servers block instead")
-
-						if d.NextArg() && d.Val() != "insecure_off" && d.Val() != "on" {
-							return nil, d.Errf("strict_sni_host only supports 'on' or 'insecure_off', got '%s'", d.Val())
-						}
-						boolVal := true
-						if d.Val() == "insecure_off" {
-							boolVal = false
-						}
-						serverOpts.StrictSNIHost = &boolVal
-
-					default:
-						return nil, d.Errf("unrecognized protocol option '%s'", d.Val())
-					}
-				}
-
-			default:
-				return nil, d.Errf("unrecognized servers option '%s'", d.Val())
+		case "log_credentials":
+			if d.NextArg() {
+				return nil, d.ArgErr()
 			}
+			serverOpts.ShouldLogCredentials = true
+
+		case "protocols":
+			protos := d.RemainingArgs()
+			for _, proto := range protos {
+				if proto != "h1" && proto != "h2" && proto != "h2c" && proto != "h3" {
+					return nil, d.Errf("unknown protocol '%s': expected h1, h2, h2c, or h3", proto)
+				}
+				if slices.Contains(serverOpts.Protocols, proto) {
+					return nil, d.Errf("protocol %s specified more than once", proto)
+				}
+				serverOpts.Protocols = append(serverOpts.Protocols, proto)
+			}
+			if nesting := d.Nesting(); d.NextBlock(nesting) {
+				return nil, d.ArgErr()
+			}
+
+		case "strict_sni_host":
+			if d.NextArg() && d.Val() != "insecure_off" && d.Val() != "on" {
+				return nil, d.Errf("strict_sni_host only supports 'on' or 'insecure_off', got '%s'", d.Val())
+			}
+			boolVal := true
+			if d.Val() == "insecure_off" {
+				boolVal = false
+			}
+			serverOpts.StrictSNIHost = &boolVal
+
+		case "trusted_proxies":
+			if !d.NextArg() {
+				return nil, d.Err("trusted_proxies expects an IP range source module name as its first argument")
+			}
+			modID := "http.ip_sources." + d.Val()
+			unm, err := caddyfile.UnmarshalModule(d, modID)
+			if err != nil {
+				return nil, err
+			}
+			source, ok := unm.(caddyhttp.IPRangeSource)
+			if !ok {
+				return nil, fmt.Errorf("module %s (%T) is not an IP range source", modID, unm)
+			}
+			jsonSource := caddyconfig.JSONModuleObject(
+				source,
+				"source",
+				source.(caddy.Module).CaddyModule().ID.Name(),
+				nil,
+			)
+			serverOpts.TrustedProxiesRaw = jsonSource
+
+		case "trusted_proxies_strict":
+			if d.NextArg() {
+				return nil, d.ArgErr()
+			}
+			serverOpts.TrustedProxiesStrict = 1
+
+		case "client_ip_headers":
+			headers := d.RemainingArgs()
+			for _, header := range headers {
+				if slices.Contains(serverOpts.ClientIPHeaders, header) {
+					return nil, d.Errf("client IP header %s specified more than once", header)
+				}
+				serverOpts.ClientIPHeaders = append(serverOpts.ClientIPHeaders, header)
+			}
+			if nesting := d.Nesting(); d.NextBlock(nesting) {
+				return nil, d.ArgErr()
+			}
+
+		case "metrics":
+			caddy.Log().Warn("The nested 'metrics' option inside `servers` is deprecated and will be removed in the next major version. Use the global 'metrics' option instead.")
+			serverOpts.Metrics = new(caddyhttp.Metrics)
+			for nesting := d.Nesting(); d.NextBlock(nesting); {
+				switch d.Val() {
+				case "per_host":
+					serverOpts.Metrics.PerHost = true
+				default:
+					return nil, d.Errf("unrecognized metrics option '%s'", d.Val())
+				}
+			}
+
+		case "trace":
+			if d.NextArg() {
+				return nil, d.ArgErr()
+			}
+			serverOpts.Trace = true
+
+		default:
+			return nil, d.Errf("unrecognized servers option '%s'", d.Val())
 		}
 	}
 	return serverOpts, nil
@@ -231,33 +268,39 @@ func unmarshalCaddyfileServerOptions(d *caddyfile.Dispenser) (any, error) {
 func applyServerOptions(
 	servers map[string]*caddyhttp.Server,
 	options map[string]any,
-	warnings *[]caddyconfig.Warning,
+	_ *[]caddyconfig.Warning,
 ) error {
 	serverOpts, ok := options["servers"].([]serverOptions)
 	if !ok {
 		return nil
 	}
 
-	for _, server := range servers {
-		// find the options that apply to this server
-		opts := func() *serverOptions {
-			for _, entry := range serverOpts {
-				if entry.ListenerAddress == "" {
-					return &entry
-				}
-				for _, listener := range server.Listen {
-					if entry.ListenerAddress == listener {
-						return &entry
-					}
-				}
-			}
-			return nil
-		}()
-
-		// if none apply, then move to the next server
-		if opts == nil {
+	// check for duplicate names, which would clobber the config
+	existingNames := map[string]bool{}
+	for _, opts := range serverOpts {
+		if opts.Name == "" {
 			continue
 		}
+		if existingNames[opts.Name] {
+			return fmt.Errorf("cannot use duplicate server name '%s'", opts.Name)
+		}
+		existingNames[opts.Name] = true
+	}
+
+	// collect the server name overrides
+	nameReplacements := map[string]string{}
+
+	for key, server := range servers {
+		// find the options that apply to this server
+		optsIndex := slices.IndexFunc(serverOpts, func(s serverOptions) bool {
+			return s.ListenerAddress == "" || slices.Contains(server.Listen, s.ListenerAddress)
+		})
+
+		// if none apply, then move to the next server
+		if optsIndex == -1 {
+			continue
+		}
+		opts := serverOpts[optsIndex]
 
 		// set all the options
 		server.ListenerWrappersRaw = opts.ListenerWrappersRaw
@@ -267,15 +310,36 @@ func applyServerOptions(
 		server.IdleTimeout = opts.IdleTimeout
 		server.KeepAliveInterval = opts.KeepAliveInterval
 		server.MaxHeaderBytes = opts.MaxHeaderBytes
+		server.EnableFullDuplex = opts.EnableFullDuplex
 		server.Protocols = opts.Protocols
 		server.StrictSNIHost = opts.StrictSNIHost
+		server.TrustedProxiesRaw = opts.TrustedProxiesRaw
+		server.ClientIPHeaders = opts.ClientIPHeaders
+		server.TrustedProxiesStrict = opts.TrustedProxiesStrict
 		server.Metrics = opts.Metrics
 		if opts.ShouldLogCredentials {
 			if server.Logs == nil {
-				server.Logs = &caddyhttp.ServerLogConfig{}
+				server.Logs = new(caddyhttp.ServerLogConfig)
 			}
 			server.Logs.ShouldLogCredentials = opts.ShouldLogCredentials
 		}
+		if opts.Trace {
+			// TODO: THIS IS EXPERIMENTAL (MAY 2024)
+			if server.Logs == nil {
+				server.Logs = new(caddyhttp.ServerLogConfig)
+			}
+			server.Logs.Trace = opts.Trace
+		}
+
+		if opts.Name != "" {
+			nameReplacements[key] = opts.Name
+		}
+	}
+
+	// rename the servers if marked to do so
+	for old, new := range nameReplacements {
+		servers[new] = servers[old]
+		delete(servers, old)
 	}
 
 	return nil

caddyconfig/httpcaddyfile/shorthands.go

@@ -0,0 +1,102 @@
+package httpcaddyfile
+
+import (
+	"regexp"
+	"strings"
+
+	"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
+)
+
+type ComplexShorthandReplacer struct {
+	search  *regexp.Regexp
+	replace string
+}
+
+type ShorthandReplacer struct {
+	complex []ComplexShorthandReplacer
+	simple  *strings.Replacer
+}
+
+func NewShorthandReplacer() ShorthandReplacer {
+	// replace shorthand placeholders (which are convenient
+	// when writing a Caddyfile) with their actual placeholder
+	// identifiers or variable names
+	replacer := strings.NewReplacer(placeholderShorthands()...)
+
+	// these are placeholders that allow a user-defined final
+	// parameters, but we still want to provide a shorthand
+	// for those, so we use a regexp to replace
+	regexpReplacements := []ComplexShorthandReplacer{
+		{regexp.MustCompile(`{header\.([\w-]*)}`), "{http.request.header.$1}"},
+		{regexp.MustCompile(`{cookie\.([\w-]*)}`), "{http.request.cookie.$1}"},
+		{regexp.MustCompile(`{labels\.([\w-]*)}`), "{http.request.host.labels.$1}"},
+		{regexp.MustCompile(`{path\.([\w-]*)}`), "{http.request.uri.path.$1}"},
+		{regexp.MustCompile(`{file\.([\w-]*)}`), "{http.request.uri.path.file.$1}"},
+		{regexp.MustCompile(`{query\.([\w-]*)}`), "{http.request.uri.query.$1}"},
+		{regexp.MustCompile(`{re\.([\w-\.]*)}`), "{http.regexp.$1}"},
+		{regexp.MustCompile(`{vars\.([\w-]*)}`), "{http.vars.$1}"},
+		{regexp.MustCompile(`{rp\.([\w-\.]*)}`), "{http.reverse_proxy.$1}"},
+		{regexp.MustCompile(`{resp\.([\w-\.]*)}`), "{http.intercept.$1}"},
+		{regexp.MustCompile(`{err\.([\w-\.]*)}`), "{http.error.$1}"},
+		{regexp.MustCompile(`{file_match\.([\w-]*)}`), "{http.matchers.file.$1}"},
+	}
+
+	return ShorthandReplacer{
+		complex: regexpReplacements,
+		simple:  replacer,
+	}
+}
+
+// placeholderShorthands returns a slice of old-new string pairs,
+// where the left of the pair is a placeholder shorthand that may
+// be used in the Caddyfile, and the right is the replacement.
+func placeholderShorthands() []string {
+	return []string{
+		"{host}", "{http.request.host}",
+		"{hostport}", "{http.request.hostport}",
+		"{port}", "{http.request.port}",
+		"{orig_method}", "{http.request.orig_method}",
+		"{orig_uri}", "{http.request.orig_uri}",
+		"{orig_path}", "{http.request.orig_uri.path}",
+		"{orig_dir}", "{http.request.orig_uri.path.dir}",
+		"{orig_file}", "{http.request.orig_uri.path.file}",
+		"{orig_query}", "{http.request.orig_uri.query}",
+		"{orig_?query}", "{http.request.orig_uri.prefixed_query}",
+		"{method}", "{http.request.method}",
+		"{uri}", "{http.request.uri}",
+		"{path}", "{http.request.uri.path}",
+		"{dir}", "{http.request.uri.path.dir}",
+		"{file}", "{http.request.uri.path.file}",
+		"{query}", "{http.request.uri.query}",
+		"{?query}", "{http.request.uri.prefixed_query}",
+		"{remote}", "{http.request.remote}",
+		"{remote_host}", "{http.request.remote.host}",
+		"{remote_port}", "{http.request.remote.port}",
+		"{scheme}", "{http.request.scheme}",
+		"{uuid}", "{http.request.uuid}",
+		"{tls_cipher}", "{http.request.tls.cipher_suite}",
+		"{tls_version}", "{http.request.tls.version}",
+		"{tls_client_fingerprint}", "{http.request.tls.client.fingerprint}",
+		"{tls_client_issuer}", "{http.request.tls.client.issuer}",
+		"{tls_client_serial}", "{http.request.tls.client.serial}",
+		"{tls_client_subject}", "{http.request.tls.client.subject}",
+		"{tls_client_certificate_pem}", "{http.request.tls.client.certificate_pem}",
+		"{tls_client_certificate_der_base64}", "{http.request.tls.client.certificate_der_base64}",
+		"{upstream_hostport}", "{http.reverse_proxy.upstream.hostport}",
+		"{client_ip}", "{http.vars.client_ip}",
+	}
+}
+
+// ApplyToSegment replaces shorthand placeholder to its full placeholder, understandable by Caddy.
+func (s ShorthandReplacer) ApplyToSegment(segment *caddyfile.Segment) {
+	if segment != nil {
+		for i := 0; i < len(*segment); i++ {
+			// simple string replacements
+			(*segment)[i].Text = s.simple.Replace((*segment)[i].Text)
+			// complex regexp replacements
+			for _, r := range s.complex {
+				(*segment)[i].Text = r.search.ReplaceAllString((*segment)[i].Text, r.replace)
+			}
+		}
+	}
+}

caddyconfig/httpcaddyfile/testdata/import_variadic.txt

@@ -0,0 +1,9 @@
+(t2) {
+    respond 200 {
+        body {args[:]}
+    }
+}
+
+:8082 {
+    import t2 false
+}

caddyconfig/httpcaddyfile/testdata/import_variadic_snippet.txt

@@ -0,0 +1,9 @@
+(t1) {
+    respond 200 {
+        body {args[:]}
+    }
+}
+
+:8081 {
+    import t1 false
+}

caddyconfig/httpcaddyfile/testdata/import_variadic_with_import.txt

@@ -0,0 +1,15 @@
+(t1) {
+    respond 200 {
+        body {args[:]}
+    }
+}
+
+:8081 {
+    import t1 false
+}
+
+import import_variadic.txt
+
+:8083 {
+    import t2 true
+}

caddyconfig/httpcaddyfile/tlsapp.go

@@ -19,16 +19,18 @@ import (
 	"encoding/json"
 	"fmt"
 	"reflect"
+	"slices"
 	"sort"
 	"strconv"
 	"strings"
 
+	"github.com/caddyserver/certmagic"
+	"github.com/mholt/acmez/v3/acme"
+
 	"github.com/caddyserver/caddy/v2"
 	"github.com/caddyserver/caddy/v2/caddyconfig"
 	"github.com/caddyserver/caddy/v2/modules/caddyhttp"
 	"github.com/caddyserver/caddy/v2/modules/caddytls"
-	"github.com/caddyserver/certmagic"
-	"github.com/mholt/acmez/acme"
 )
 
 func (st ServerType) buildTLSApp(
@@ -36,7 +38,6 @@ func (st ServerType) buildTLSApp(
 	options map[string]any,
 	warnings []caddyconfig.Warning,
 ) (*caddytls.TLS, []caddyconfig.Warning, error) {
-
 	tlsApp := &caddytls.TLS{CertificatesRaw: make(caddy.ModuleMap)}
 	var certLoaders []caddytls.CertificateLoader
 
@@ -44,29 +45,26 @@ func (st ServerType) buildTLSApp(
 	if hp, ok := options["http_port"].(int); ok {
 		httpPort = strconv.Itoa(hp)
 	}
-	httpsPort := strconv.Itoa(caddyhttp.DefaultHTTPSPort)
-	if hsp, ok := options["https_port"].(int); ok {
-		httpsPort = strconv.Itoa(hsp)
+	autoHTTPS := []string{}
+	if ah, ok := options["auto_https"].([]string); ok {
+		autoHTTPS = ah
 	}
 
-	// count how many server blocks have a TLS-enabled key with
-	// no host, and find all hosts that share a server block with
-	// a hostless key, so that they don't get forgotten/omitted
-	// by auto-HTTPS (since they won't appear in route matchers)
-	var serverBlocksWithTLSHostlessKey int
+	// find all hosts that share a server block with a hostless
+	// key, so that they don't get forgotten/omitted by auto-HTTPS
+	// (since they won't appear in route matchers)
 	httpsHostsSharedWithHostlessKey := make(map[string]struct{})
-	for _, pair := range pairings {
-		for _, sb := range pair.serverBlocks {
-			for _, addr := range sb.keys {
-				if addr.Host == "" {
-					// this address has no hostname, but if it's explicitly set
-					// to HTTPS, then we need to count it as being TLS-enabled
-					if addr.Scheme == "https" || addr.Port == httpsPort {
-						serverBlocksWithTLSHostlessKey++
+	if !slices.Contains(autoHTTPS, "off") {
+		for _, pair := range pairings {
+			for _, sb := range pair.serverBlocks {
+				for _, addr := range sb.parsedKeys {
+					if addr.Host != "" {
+						continue
 					}
+
 					// this server block has a hostless key, now
 					// go through and add all the hosts to the set
-					for _, otherAddr := range sb.keys {
+					for _, otherAddr := range sb.parsedKeys {
 						if otherAddr.Original == addr.Original {
 							continue
 						}
@@ -94,9 +92,33 @@ func (st ServerType) buildTLSApp(
 		tlsApp.Automation.Policies = append(tlsApp.Automation.Policies, catchAllAP)
 	}
 
+	var wildcardHosts []string                        // collect all hosts that have a wildcard in them, and aren't HTTP
+	forcedAutomatedNames := make(map[string]struct{}) // explicitly configured to be automated, even if covered by a wildcard
+
+	for _, p := range pairings {
+		var addresses []string
+		for _, addressWithProtocols := range p.addressesWithProtocols {
+			addresses = append(addresses, addressWithProtocols.address)
+		}
+		if !listenersUseAnyPortOtherThan(addresses, httpPort) {
+			continue
+		}
+		for _, sblock := range p.serverBlocks {
+			for _, addr := range sblock.parsedKeys {
+				if strings.HasPrefix(addr.Host, "*.") {
+					wildcardHosts = append(wildcardHosts, addr.Host[2:])
+				}
+			}
+		}
+	}
+
 	for _, p := range pairings {
 		// avoid setting up TLS automation policies for a server that is HTTP-only
-		if !listenersUseAnyPortOtherThan(p.addresses, httpPort) {
+		var addresses []string
+		for _, addressWithProtocols := range p.addressesWithProtocols {
+			addresses = append(addresses, addressWithProtocols.address)
+		}
+		if !listenersUseAnyPortOtherThan(addresses, httpPort) {
 			continue
 		}
 
@@ -113,6 +135,12 @@ func (st ServerType) buildTLSApp(
 				return nil, warnings, err
 			}
 
+			// make a plain copy so we can compare whether we made any changes
+			apCopy, err := newBaseAutomationPolicy(options, warnings, true)
+			if err != nil {
+				return nil, warnings, err
+			}
+
 			sblockHosts := sblock.hostsFromKeys(false)
 			if len(sblockHosts) == 0 && catchAllAP != nil {
 				ap = catchAllAP
@@ -123,6 +151,18 @@ func (st ServerType) buildTLSApp(
 				ap.OnDemand = true
 			}
 
+			// collect hosts that are forced to have certs automated for their specific name
+			if _, ok := sblock.pile["tls.force_automate"]; ok {
+				for _, host := range sblockHosts {
+					forcedAutomatedNames[host] = struct{}{}
+				}
+			}
+
+			// reuse private keys tls
+			if _, ok := sblock.pile["tls.reuse_private_keys"]; ok {
+				ap.ReusePrivateKeys = true
+			}
+
 			if keyTypeVals, ok := sblock.pile["tls.key_type"]; ok {
 				ap.KeyType = keyTypeVals[0].Value.(string)
 			}
@@ -134,6 +174,19 @@ func (st ServerType) buildTLSApp(
 					issuers = append(issuers, issuerVal.Value.(certmagic.Issuer))
 				}
 				if ap == catchAllAP && !reflect.DeepEqual(ap.Issuers, issuers) {
+					// this more correctly implements an error check that was removed
+					// below; try it with this config:
+					//
+					// :443 {
+					// 	bind 127.0.0.1
+					// }
+					//
+					// :443 {
+					// 	bind ::1
+					// 	tls {
+					// 		issuer acme
+					// 	}
+					// }
 					return nil, warnings, fmt.Errorf("automation policy from site block is also default/catch-all policy because of key without hostname, and the two are in conflict: %#v != %#v", ap.Issuers, issuers)
 				}
 				ap.Issuers = issuers
@@ -168,42 +221,50 @@ func (st ServerType) buildTLSApp(
 					if acmeIssuer.Challenges.BindHost == "" {
 						// only binding to one host is supported
 						var bindHost string
-						if bindHosts, ok := cfgVal.Value.([]string); ok && len(bindHosts) > 0 {
-							bindHost = bindHosts[0]
+						if asserted, ok := cfgVal.Value.(addressesWithProtocols); ok && len(asserted.addresses) > 0 {
+							bindHost = asserted.addresses[0]
 						}
 						acmeIssuer.Challenges.BindHost = bindHost
 					}
 				}
 			}
 
-			// first make sure this block is allowed to create an automation policy;
-			// doing so is forbidden if it has a key with no host (i.e. ":443")
+			// we used to ensure this block is allowed to create an automation policy;
+			// doing so was forbidden if it has a key with no host (i.e. ":443")
 			// and if there is a different server block that also has a key with no
 			// host -- since a key with no host matches any host, we need its
 			// associated automation policy to have an empty Subjects list, i.e. no
 			// host filter, which is indistinguishable between the two server blocks
 			// because automation is not done in the context of a particular server...
 			// this is an example of a poor mapping from Caddyfile to JSON but that's
-			// the least-leaky abstraction I could figure out
-			if len(sblockHosts) == 0 {
-				if serverBlocksWithTLSHostlessKey > 1 {
-					// this server block and at least one other has a key with no host,
-					// making the two indistinguishable; it is misleading to define such
-					// a policy within one server block since it actually will apply to
-					// others as well
-					return nil, warnings, fmt.Errorf("cannot make a TLS automation policy from a server block that has a host-less address when there are other TLS-enabled server block addresses lacking a host")
-				}
-				if catchAllAP == nil {
-					// this server block has a key with no hosts, but there is not yet
-					// a catch-all automation policy (probably because no global options
-					// were set), so this one becomes it
-					catchAllAP = ap
+			// the least-leaky abstraction I could figure out -- however, this check
+			// was preventing certain listeners, like those provided by plugins, from
+			// being used as desired (see the Tailscale listener plugin), so I removed
+			// the check: and I think since I originally wrote the check I added a new
+			// check above which *properly* detects this ambiguity without breaking the
+			// listener plugin; see the check above with a commented example config
+			if len(sblockHosts) == 0 && catchAllAP == nil {
+				// this server block has a key with no hosts, but there is not yet
+				// a catch-all automation policy (probably because no global options
+				// were set), so this one becomes it
+				catchAllAP = ap
+			}
+
+			hostsNotHTTP := sblock.hostsFromKeysNotHTTP(httpPort)
+			sort.Strings(hostsNotHTTP) // solely for deterministic test results
+
+			// if the we prefer wildcards and the AP is unchanged,
+			// then we can skip this AP because it should be covered
+			// by an AP with a wildcard
+			if slices.Contains(autoHTTPS, "prefer_wildcard") {
+				if hostsCoveredByWildcard(hostsNotHTTP, wildcardHosts) &&
+					reflect.DeepEqual(ap, apCopy) {
+					continue
 				}
 			}
 
 			// associate our new automation policy with this server block's hosts
-			ap.Subjects = sblock.hostsFromKeysNotHTTP(httpPort)
-			sort.Strings(ap.Subjects) // solely for deterministic test results
+			ap.SubjectsRaw = hostsNotHTTP
 
 			// if a combination of public and internal names were given
 			// for this same server block and no issuer was specified, we
@@ -213,7 +274,11 @@ func (st ServerType) buildTLSApp(
 			var ap2 *caddytls.AutomationPolicy
 			if len(ap.Issuers) == 0 {
 				var internal, external []string
-				for _, s := range ap.Subjects {
+				for _, s := range ap.SubjectsRaw {
+					// do not create Issuers for Tailscale domains; they will be given a Manager instead
+					if isTailscaleDomain(s) {
+						continue
+					}
 					if !certmagic.SubjectQualifiesForCert(s) {
 						return nil, warnings, fmt.Errorf("subject does not qualify for certificate: '%s'", s)
 					}
@@ -231,13 +296,14 @@ func (st ServerType) buildTLSApp(
 					}
 				}
 				if len(external) > 0 && len(internal) > 0 {
-					ap.Subjects = external
+					ap.SubjectsRaw = external
 					apCopy := *ap
 					ap2 = &apCopy
-					ap2.Subjects = internal
+					ap2.SubjectsRaw = internal
 					ap2.IssuersRaw = []json.RawMessage{caddyconfig.JSONModuleObject(caddytls.InternalIssuer{}, "module", "internal", &warnings)}
 				}
 			}
+
 			if tlsApp.Automation == nil {
 				tlsApp.Automation = new(caddytls.AutomationConfig)
 			}
@@ -272,7 +338,7 @@ func (st ServerType) buildTLSApp(
 					combined = reflect.New(reflect.TypeOf(cl)).Elem()
 				}
 				clVal := reflect.ValueOf(cl)
-				for i := 0; i < clVal.Len(); i++ {
+				for i := range clVal.Len() {
 					combined = reflect.Append(combined, clVal.Index(i))
 				}
 				loadersByName[name] = combined.Interface().(caddytls.CertificateLoader)
@@ -291,6 +357,42 @@ func (st ServerType) buildTLSApp(
 		tlsApp.Automation.OnDemand = onDemand
 	}
 
+	// set up "global" (to the TLS app) DNS provider config
+	if globalDNS, ok := options["dns"]; ok && globalDNS != nil {
+		tlsApp.DNSRaw = caddyconfig.JSONModuleObject(globalDNS, "name", globalDNS.(caddy.Module).CaddyModule().ID.Name(), nil)
+	}
+
+	// set up ECH from Caddyfile options
+	if ech, ok := options["ech"].(*caddytls.ECH); ok {
+		tlsApp.EncryptedClientHello = ech
+
+		// outer server names will need certificates, so make sure they're included
+		// in an automation policy for them that applies any global options
+		ap, err := newBaseAutomationPolicy(options, warnings, true)
+		if err != nil {
+			return nil, warnings, err
+		}
+		for _, cfg := range ech.Configs {
+			if cfg.PublicName != "" {
+				ap.SubjectsRaw = append(ap.SubjectsRaw, cfg.PublicName)
+			}
+		}
+		if tlsApp.Automation == nil {
+			tlsApp.Automation = new(caddytls.AutomationConfig)
+		}
+		tlsApp.Automation.Policies = append(tlsApp.Automation.Policies, ap)
+	}
+
+	// if the storage clean interval is a boolean, then it's "off" to disable cleaning
+	if sc, ok := options["storage_check"].(string); ok && sc == "off" {
+		tlsApp.DisableStorageCheck = true
+	}
+
+	// if the storage clean interval is a boolean, then it's "off" to disable cleaning
+	if sci, ok := options["storage_clean_interval"].(bool); ok && !sci {
+		tlsApp.DisableStorageClean = true
+	}
+
 	// set the storage clean interval if configured
 	if storageCleanInterval, ok := options["storage_clean_interval"].(caddy.Duration); ok {
 		if tlsApp.Automation == nil {
@@ -331,16 +433,25 @@ func (st ServerType) buildTLSApp(
 	internalAP := &caddytls.AutomationPolicy{
 		IssuersRaw: []json.RawMessage{json.RawMessage(`{"module":"internal"}`)},
 	}
-	for h := range httpsHostsSharedWithHostlessKey {
-		al = append(al, h)
-		if !certmagic.SubjectQualifiesForPublicCert(h) {
-			internalAP.Subjects = append(internalAP.Subjects, h)
+	if !slices.Contains(autoHTTPS, "off") && !slices.Contains(autoHTTPS, "disable_certs") {
+		for h := range httpsHostsSharedWithHostlessKey {
+			al = append(al, h)
+			if !certmagic.SubjectQualifiesForPublicCert(h) {
+				internalAP.SubjectsRaw = append(internalAP.SubjectsRaw, h)
+			}
 		}
 	}
+	for name := range forcedAutomatedNames {
+		if slices.Contains(al, name) {
+			continue
+		}
+		al = append(al, name)
+	}
+	slices.Sort(al) // to stabilize the adapt output
 	if len(al) > 0 {
 		tlsApp.CertificatesRaw["automate"] = caddyconfig.JSON(al, &warnings)
 	}
-	if len(internalAP.Subjects) > 0 {
+	if len(internalAP.SubjectsRaw) > 0 {
 		if tlsApp.Automation == nil {
 			tlsApp.Automation = new(caddytls.AutomationConfig)
 		}
@@ -353,25 +464,22 @@ func (st ServerType) buildTLSApp(
 		globalEmail := options["email"]
 		globalACMECA := options["acme_ca"]
 		globalACMECARoot := options["acme_ca_root"]
-		globalACMEDNS := options["acme_dns"]
+		_, globalACMEDNS := options["acme_dns"] // can be set to nil (to use globally-defined "dns" value instead), but it is still set
 		globalACMEEAB := options["acme_eab"]
 		globalPreferredChains := options["preferred_chains"]
-		hasGlobalACMEDefaults := globalEmail != nil || globalACMECA != nil || globalACMECARoot != nil || globalACMEDNS != nil || globalACMEEAB != nil || globalPreferredChains != nil
+		hasGlobalACMEDefaults := globalEmail != nil || globalACMECA != nil || globalACMECARoot != nil || globalACMEDNS || globalACMEEAB != nil || globalPreferredChains != nil
 		if hasGlobalACMEDefaults {
-			for i := 0; i < len(tlsApp.Automation.Policies); i++ {
+			for i := range tlsApp.Automation.Policies {
 				ap := tlsApp.Automation.Policies[i]
 				if len(ap.Issuers) == 0 && automationPolicyHasAllPublicNames(ap) {
 					// for public names, create default issuers which will later be filled in with configured global defaults
 					// (internal names will implicitly use the internal issuer at auto-https time)
-					ap.Issuers = caddytls.DefaultIssuers()
+					emailStr, _ := globalEmail.(string)
+					ap.Issuers = caddytls.DefaultIssuers(emailStr)
 
 					// if a specific endpoint is configured, can't use multiple default issuers
 					if globalACMECA != nil {
-						if strings.Contains(globalACMECA.(string), "zerossl") {
-							ap.Issuers = []certmagic.Issuer{&caddytls.ZeroSSLIssuer{ACMEIssuer: new(caddytls.ACMEIssuer)}}
-						} else {
-							ap.Issuers = []certmagic.Issuer{new(caddytls.ACMEIssuer)}
-						}
+						ap.Issuers = []certmagic.Issuer{new(caddytls.ACMEIssuer)}
 					}
 				}
 			}
@@ -406,7 +514,7 @@ func (st ServerType) buildTLSApp(
 		// for convenience)
 		automationHostSet := make(map[string]struct{})
 		for _, ap := range tlsApp.Automation.Policies {
-			for _, s := range ap.Subjects {
+			for _, s := range ap.SubjectsRaw {
 				if _, ok := automationHostSet[s]; ok {
 					return nil, warnings, fmt.Errorf("hostname appears in more than one automation policy, making certificate management ambiguous: %s", s)
 				}
@@ -441,9 +549,11 @@ func fillInGlobalACMEDefaults(issuer certmagic.Issuer, options map[string]any) e
 	globalEmail := options["email"]
 	globalACMECA := options["acme_ca"]
 	globalACMECARoot := options["acme_ca_root"]
-	globalACMEDNS := options["acme_dns"]
+	globalACMEDNS, globalACMEDNSok := options["acme_dns"] // can be set to nil (to use globally-defined "dns" value instead), but it is still set
 	globalACMEEAB := options["acme_eab"]
 	globalPreferredChains := options["preferred_chains"]
+	globalCertLifetime := options["cert_lifetime"]
+	globalHTTPPort, globalHTTPSPort := options["http_port"], options["https_port"]
 
 	if globalEmail != nil && acmeIssuer.Email == "" {
 		acmeIssuer.Email = globalEmail.(string)
@@ -451,10 +561,16 @@ func fillInGlobalACMEDefaults(issuer certmagic.Issuer, options map[string]any) e
 	if globalACMECA != nil && acmeIssuer.CA == "" {
 		acmeIssuer.CA = globalACMECA.(string)
 	}
-	if globalACMECARoot != nil && !sliceContains(acmeIssuer.TrustedRootsPEMFiles, globalACMECARoot.(string)) {
+	if globalACMECARoot != nil && !slices.Contains(acmeIssuer.TrustedRootsPEMFiles, globalACMECARoot.(string)) {
 		acmeIssuer.TrustedRootsPEMFiles = append(acmeIssuer.TrustedRootsPEMFiles, globalACMECARoot.(string))
 	}
-	if globalACMEDNS != nil && (acmeIssuer.Challenges == nil || acmeIssuer.Challenges.DNS == nil) {
+	if globalACMEDNSok && (acmeIssuer.Challenges == nil || acmeIssuer.Challenges.DNS == nil) {
+		if globalACMEDNS == nil {
+			globalACMEDNS = options["dns"]
+			if globalACMEDNS == nil {
+				return fmt.Errorf("acme_dns specified without DNS provider config, but no provider specified with 'dns' global option")
+			}
+		}
 		acmeIssuer.Challenges = &caddytls.ChallengesConfig{
 			DNS: &caddytls.DNSChallengeConfig{
 				ProviderRaw: caddyconfig.JSONModuleObject(globalACMEDNS, "name", globalACMEDNS.(caddy.Module).CaddyModule().ID.Name(), nil),
@@ -467,6 +583,28 @@ func fillInGlobalACMEDefaults(issuer certmagic.Issuer, options map[string]any) e
 	if globalPreferredChains != nil && acmeIssuer.PreferredChains == nil {
 		acmeIssuer.PreferredChains = globalPreferredChains.(*caddytls.ChainPreference)
 	}
+	// only configure alt HTTP and TLS-ALPN ports if the DNS challenge is not enabled (wouldn't hurt, but isn't necessary since the DNS challenge is exclusive of others)
+	if globalHTTPPort != nil && (acmeIssuer.Challenges == nil || acmeIssuer.Challenges.DNS == nil) && (acmeIssuer.Challenges == nil || acmeIssuer.Challenges.HTTP == nil || acmeIssuer.Challenges.HTTP.AlternatePort == 0) {
+		if acmeIssuer.Challenges == nil {
+			acmeIssuer.Challenges = new(caddytls.ChallengesConfig)
+		}
+		if acmeIssuer.Challenges.HTTP == nil {
+			acmeIssuer.Challenges.HTTP = new(caddytls.HTTPChallengeConfig)
+		}
+		acmeIssuer.Challenges.HTTP.AlternatePort = globalHTTPPort.(int)
+	}
+	if globalHTTPSPort != nil && (acmeIssuer.Challenges == nil || acmeIssuer.Challenges.DNS == nil) && (acmeIssuer.Challenges == nil || acmeIssuer.Challenges.TLSALPN == nil || acmeIssuer.Challenges.TLSALPN.AlternatePort == 0) {
+		if acmeIssuer.Challenges == nil {
+			acmeIssuer.Challenges = new(caddytls.ChallengesConfig)
+		}
+		if acmeIssuer.Challenges.TLSALPN == nil {
+			acmeIssuer.Challenges.TLSALPN = new(caddytls.TLSALPNChallengeConfig)
+		}
+		acmeIssuer.Challenges.TLSALPN.AlternatePort = globalHTTPSPort.(int)
+	}
+	if globalCertLifetime != nil && acmeIssuer.CertificateLifetime == 0 {
+		acmeIssuer.CertificateLifetime = globalCertLifetime.(caddy.Duration)
+	}
 	return nil
 }
 
@@ -475,7 +613,11 @@ func fillInGlobalACMEDefaults(issuer certmagic.Issuer, options map[string]any) e
 // for any other automation policies. A nil policy (and no error) will be
 // returned if there are no default/global options. However, if always is
 // true, a non-nil value will always be returned (unless there is an error).
-func newBaseAutomationPolicy(options map[string]any, warnings []caddyconfig.Warning, always bool) (*caddytls.AutomationPolicy, error) {
+func newBaseAutomationPolicy(
+	options map[string]any,
+	_ []caddyconfig.Warning,
+	always bool,
+) (*caddytls.AutomationPolicy, error) {
 	issuers, hasIssuers := options["cert_issuer"]
 	_, hasLocalCerts := options["local_certs"]
 	keyType, hasKeyType := options["key_type"]
@@ -527,7 +669,7 @@ func consolidateAutomationPolicies(aps []*caddytls.AutomationPolicy) []*caddytls
 		if automationPolicyIsSubset(aps[j], aps[i]) {
 			return false
 		}
-		return len(aps[i].Subjects) > len(aps[j].Subjects)
+		return len(aps[i].SubjectsRaw) > len(aps[j].SubjectsRaw)
 	})
 
 	emptyAPCount := 0
@@ -535,13 +677,13 @@ func consolidateAutomationPolicies(aps []*caddytls.AutomationPolicy) []*caddytls
 	// compute the number of empty policies (disregarding subjects) - see #4128
 	emptyAP := new(caddytls.AutomationPolicy)
 	for i := 0; i < len(aps); i++ {
-		emptyAP.Subjects = aps[i].Subjects
+		emptyAP.SubjectsRaw = aps[i].SubjectsRaw
 		if reflect.DeepEqual(aps[i], emptyAP) {
 			emptyAPCount++
 			if !automationPolicyHasAllPublicNames(aps[i]) {
 				// if this automation policy has internal names, we might as well remove it
 				// so auto-https can implicitly use the internal issuer
-				aps = append(aps[:i], aps[i+1:]...)
+				aps = slices.Delete(aps, i, i+1)
 				i--
 			}
 		}
@@ -558,7 +700,7 @@ outer:
 		for j := i + 1; j < len(aps); j++ {
 			// if they're exactly equal in every way, just keep one of them
 			if reflect.DeepEqual(aps[i], aps[j]) {
-				aps = append(aps[:j], aps[j+1:]...)
+				aps = slices.Delete(aps, j, j+1)
 				// must re-evaluate current i against next j; can't skip it!
 				// even if i decrements to -1, will be incremented to 0 immediately
 				i--
@@ -572,12 +714,14 @@ outer:
 			// eaten up by the one with subjects; and if both have subjects, we
 			// need to combine their lists
 			if reflect.DeepEqual(aps[i].IssuersRaw, aps[j].IssuersRaw) &&
+				reflect.DeepEqual(aps[i].ManagersRaw, aps[j].ManagersRaw) &&
 				bytes.Equal(aps[i].StorageRaw, aps[j].StorageRaw) &&
 				aps[i].MustStaple == aps[j].MustStaple &&
 				aps[i].KeyType == aps[j].KeyType &&
 				aps[i].OnDemand == aps[j].OnDemand &&
+				aps[i].ReusePrivateKeys == aps[j].ReusePrivateKeys &&
 				aps[i].RenewalWindowRatio == aps[j].RenewalWindowRatio {
-				if len(aps[i].Subjects) > 0 && len(aps[j].Subjects) == 0 {
+				if len(aps[i].SubjectsRaw) > 0 && len(aps[j].SubjectsRaw) == 0 {
 					// later policy (at j) has no subjects ("catch-all"), so we can
 					// remove the identical-but-more-specific policy that comes first
 					// AS LONG AS it is not shadowed by another policy before it; e.g.
@@ -586,18 +730,18 @@ outer:
 					// cause example.com to be served by the less specific policy for
 					// '*.com', which might be different (yes we've seen this happen)
 					if automationPolicyShadows(i, aps) >= j {
-						aps = append(aps[:i], aps[i+1:]...)
+						aps = slices.Delete(aps, i, i+1)
 						i--
 						continue outer
 					}
 				} else {
 					// avoid repeated subjects
-					for _, subj := range aps[j].Subjects {
-						if !sliceContains(aps[i].Subjects, subj) {
-							aps[i].Subjects = append(aps[i].Subjects, subj)
+					for _, subj := range aps[j].SubjectsRaw {
+						if !slices.Contains(aps[i].SubjectsRaw, subj) {
+							aps[i].SubjectsRaw = append(aps[i].SubjectsRaw, subj)
 						}
 					}
-					aps = append(aps[:j], aps[j+1:]...)
+					aps = slices.Delete(aps, j, j+1)
 					j--
 				}
 			}
@@ -610,20 +754,16 @@ outer:
 // automationPolicyIsSubset returns true if a's subjects are a subset
 // of b's subjects.
 func automationPolicyIsSubset(a, b *caddytls.AutomationPolicy) bool {
-	if len(b.Subjects) == 0 {
+	if len(b.SubjectsRaw) == 0 {
 		return true
 	}
-	if len(a.Subjects) == 0 {
+	if len(a.SubjectsRaw) == 0 {
 		return false
 	}
-	for _, aSubj := range a.Subjects {
-		var inSuperset bool
-		for _, bSubj := range b.Subjects {
-			if certmagic.MatchWildcard(aSubj, bSubj) {
-				inSuperset = true
-				break
-			}
-		}
+	for _, aSubj := range a.SubjectsRaw {
+		inSuperset := slices.ContainsFunc(b.SubjectsRaw, func(bSubj string) bool {
+			return certmagic.MatchWildcard(aSubj, bSubj)
+		})
 		if !inSuperset {
 			return false
 		}
@@ -649,17 +789,47 @@ func automationPolicyShadows(i int, aps []*caddytls.AutomationPolicy) int {
 // subjectQualifiesForPublicCert is like certmagic.SubjectQualifiesForPublicCert() except
 // that this allows domains with multiple wildcard levels like '*.*.example.com' to qualify
 // if the automation policy has OnDemand enabled (i.e. this function is more lenient).
+//
+// IP subjects are considered as non-qualifying for public certs. Technically, there are
+// now public ACME CAs as well as non-ACME CAs that issue IP certificates. But this function
+// is used solely for implicit automation (defaults), where it gets really complicated to
+// keep track of which issuers support IP certificates in which circumstances. Currently,
+// issuers that support IP certificates are very few, and all require some sort of config
+// from the user anyway (such as an account credential). Since we cannot implicitly and
+// automatically get public IP certs without configuration from the user, we treat IPs as
+// not qualifying for public certificates. Users should expressly configure an issuer
+// that supports IP certs for that purpose.
 func subjectQualifiesForPublicCert(ap *caddytls.AutomationPolicy, subj string) bool {
 	return !certmagic.SubjectIsIP(subj) &&
 		!certmagic.SubjectIsInternal(subj) &&
 		(strings.Count(subj, "*.") < 2 || ap.OnDemand)
 }
 
+// automationPolicyHasAllPublicNames returns true if all the names on the policy
+// do NOT qualify for public certs OR are tailscale domains.
 func automationPolicyHasAllPublicNames(ap *caddytls.AutomationPolicy) bool {
-	for _, subj := range ap.Subjects {
-		if !subjectQualifiesForPublicCert(ap, subj) {
-			return false
+	return !slices.ContainsFunc(ap.SubjectsRaw, func(i string) bool {
+		return !subjectQualifiesForPublicCert(ap, i) || isTailscaleDomain(i)
+	})
+}
+
+func isTailscaleDomain(name string) bool {
+	return strings.HasSuffix(strings.ToLower(name), ".ts.net")
+}
+
+func hostsCoveredByWildcard(hosts []string, wildcards []string) bool {
+	if len(hosts) == 0 || len(wildcards) == 0 {
+		return false
+	}
+	for _, host := range hosts {
+		for _, wildcard := range wildcards {
+			if strings.HasPrefix(host, "*.") {
+				continue
+			}
+			if certmagic.MatchWildcard(host, "*."+wildcard) {
+				return true
+			}
 		}
 	}
-	return true
+	return false
 }

caddyconfig/httpcaddyfile/tlsapp_test.go

@@ -47,8 +47,8 @@ func TestAutomationPolicyIsSubset(t *testing.T) {
 			expect: false,
 		},
 	} {
-		apA := &caddytls.AutomationPolicy{Subjects: test.a}
-		apB := &caddytls.AutomationPolicy{Subjects: test.b}
+		apA := &caddytls.AutomationPolicy{SubjectsRaw: test.a}
+		apB := &caddytls.AutomationPolicy{SubjectsRaw: test.b}
 		if actual := automationPolicyIsSubset(apA, apB); actual != test.expect {
 			t.Errorf("Test %d: Expected %t but got %t (A: %v  B: %v)", i, test.expect, actual, test.a, test.b)
 		}

caddyconfig/httploader.go

@@ -30,8 +30,14 @@ func init() {
 	caddy.RegisterModule(HTTPLoader{})
 }
 
-// HTTPLoader can load Caddy configs over HTTP(S). It can adapt the config
-// based on the Content-Type header of the HTTP response.
+// HTTPLoader can load Caddy configs over HTTP(S).
+//
+// If the response is not a JSON config, a config adapter must be specified
+// either in the loader config (`adapter`), or in the Content-Type HTTP header
+// returned in the HTTP response from the server. The Content-Type header is
+// read just like the admin API's `/load` endpoint. If you don't have control
+// over the HTTP server (but can still trust its response), you can override
+// the Content-Type header by setting the `adapter` property in this config.
 type HTTPLoader struct {
 	// The method for the request. Default: GET
 	Method string `json:"method,omitempty"`
@@ -45,6 +51,11 @@ type HTTPLoader struct {
 	// Maximum time allowed for a complete connection and request.
 	Timeout caddy.Duration `json:"timeout,omitempty"`
 
+	// The name of the config adapter to use, if any. Only needed
+	// if the HTTP response is not a JSON config and if the server's
+	// Content-Type header is missing or incorrect.
+	Adapter string `json:"adapter,omitempty"`
+
 	TLS *struct {
 		// Present this instance's managed remote identity credentials to the server.
 		UseServerIdentity bool `json:"use_server_identity,omitempty"`
@@ -94,7 +105,7 @@ func (hl HTTPLoader) LoadConfig(ctx caddy.Context) ([]byte, error) {
 		}
 	}
 
-	resp, err := client.Do(req)
+	resp, err := doHttpCallWithRetries(ctx, client, req)
 	if err != nil {
 		return nil, err
 	}
@@ -108,7 +119,12 @@ func (hl HTTPLoader) LoadConfig(ctx caddy.Context) ([]byte, error) {
 		return nil, err
 	}
 
-	result, warnings, err := adaptByContentType(resp.Header.Get("Content-Type"), body)
+	// adapt the config based on either manually-configured adapter or server's response header
+	ct := resp.Header.Get("Content-Type")
+	if hl.Adapter != "" {
+		ct = "text/" + hl.Adapter
+	}
+	result, warnings, err := adaptByContentType(ct, body)
 	if err != nil {
 		return nil, err
 	}
@@ -119,6 +135,38 @@ func (hl HTTPLoader) LoadConfig(ctx caddy.Context) ([]byte, error) {
 	return result, nil
 }
 
+func attemptHttpCall(client *http.Client, request *http.Request) (*http.Response, error) {
+	resp, err := client.Do(request)
+	if err != nil {
+		return nil, fmt.Errorf("problem calling http loader url: %v", err)
+	} else if resp.StatusCode < 200 || resp.StatusCode > 499 {
+		resp.Body.Close()
+		return nil, fmt.Errorf("bad response status code from http loader url: %v", resp.StatusCode)
+	}
+	return resp, nil
+}
+
+func doHttpCallWithRetries(ctx caddy.Context, client *http.Client, request *http.Request) (*http.Response, error) {
+	var resp *http.Response
+	var err error
+	const maxAttempts = 10
+
+	for i := 0; i < maxAttempts; i++ {
+		resp, err = attemptHttpCall(client, request)
+		if err != nil && i < maxAttempts-1 {
+			select {
+			case <-time.After(time.Millisecond * 500):
+			case <-ctx.Done():
+				return resp, ctx.Err()
+			}
+		} else {
+			break
+		}
+	}
+
+	return resp, err
+}
+
 func (hl HTTPLoader) makeClient(ctx caddy.Context) (*http.Client, error) {
 	client := &http.Client{
 		Timeout: time.Duration(hl.Timeout),
@@ -133,19 +181,16 @@ func (hl HTTPLoader) makeClient(ctx caddy.Context) (*http.Client, error) {
 			if err != nil {
 				return nil, fmt.Errorf("getting server identity credentials: %v", err)
 			}
-			if tlsConfig == nil {
-				tlsConfig = new(tls.Config)
-			}
-			tlsConfig.Certificates = certs
+			// See https://github.com/securego/gosec/issues/1054#issuecomment-2072235199
+			//nolint:gosec
+			tlsConfig = &tls.Config{Certificates: certs}
 		} else if hl.TLS.ClientCertificateFile != "" && hl.TLS.ClientCertificateKeyFile != "" {
 			cert, err := tls.LoadX509KeyPair(hl.TLS.ClientCertificateFile, hl.TLS.ClientCertificateKeyFile)
 			if err != nil {
 				return nil, err
 			}
-			if tlsConfig == nil {
-				tlsConfig = new(tls.Config)
-			}
-			tlsConfig.Certificates = []tls.Certificate{cert}
+			//nolint:gosec
+			tlsConfig = &tls.Config{Certificates: []tls.Certificate{cert}}
 		}
 
 		// trusted server certs

caddytest/caddytest.go

@@ -8,6 +8,7 @@ import (
 	"errors"
 	"fmt"
 	"io"
+	"io/fs"
 	"log"
 	"net"
 	"net/http"
@@ -22,19 +23,20 @@ import (
 	"time"
 
 	"github.com/aryann/difflib"
-	"github.com/caddyserver/caddy/v2/caddyconfig"
+
 	caddycmd "github.com/caddyserver/caddy/v2/cmd"
 
+	"github.com/caddyserver/caddy/v2/caddyconfig"
 	// plug in Caddy modules here
 	_ "github.com/caddyserver/caddy/v2/modules/standard"
 )
 
-// Defaults store any configuration required to make the tests run
-type Defaults struct {
+// Config store any configuration required to make the tests run
+type Config struct {
 	// Port we expect caddy to listening on
 	AdminPort int
 	// Certificates we expect to be loaded before attempting to run the tests
-	Certifcates []string
+	Certificates []string
 	// TestRequestTimeout is the time to wait for a http request to
 	TestRequestTimeout time.Duration
 	// LoadRequestTimeout is the time to wait for the config to be loaded against the caddy server
@@ -42,9 +44,9 @@ type Defaults struct {
 }
 
 // Default testing values
-var Default = Defaults{
+var Default = Config{
 	AdminPort:          2999, // different from what a real server also running on a developer's machine might be
-	Certifcates:        []string{"/caddy.localhost.crt", "/caddy.localhost.key"},
+	Certificates:       []string{"/caddy.localhost.crt", "/caddy.localhost.key"},
 	TestRequestTimeout: 5 * time.Second,
 	LoadRequestTimeout: 5 * time.Second,
 }
@@ -58,12 +60,12 @@ var (
 type Tester struct {
 	Client       *http.Client
 	configLoaded bool
-	t            *testing.T
+	t            testing.TB
+	config       Config
 }
 
 // NewTester will create a new testing client with an attached cookie jar
-func NewTester(t *testing.T) *Tester {
-
+func NewTester(t testing.TB) *Tester {
 	jar, err := cookiejar.New(nil)
 	if err != nil {
 		t.Fatalf("failed to create cookiejar: %s", err)
@@ -77,9 +79,29 @@ func NewTester(t *testing.T) *Tester {
 		},
 		configLoaded: false,
 		t:            t,
+		config:       Default,
 	}
 }
 
+// WithDefaultOverrides this will override the default test configuration with the provided values.
+func (tc *Tester) WithDefaultOverrides(overrides Config) *Tester {
+	if overrides.AdminPort != 0 {
+		tc.config.AdminPort = overrides.AdminPort
+	}
+	if len(overrides.Certificates) > 0 {
+		tc.config.Certificates = overrides.Certificates
+	}
+	if overrides.TestRequestTimeout != 0 {
+		tc.config.TestRequestTimeout = overrides.TestRequestTimeout
+		tc.Client.Timeout = overrides.TestRequestTimeout
+	}
+	if overrides.LoadRequestTimeout != 0 {
+		tc.config.LoadRequestTimeout = overrides.LoadRequestTimeout
+	}
+
+	return tc
+}
+
 type configLoadError struct {
 	Response string
 }
@@ -94,7 +116,6 @@ func timeElapsed(start time.Time, name string) {
 // InitServer this will configure the server with a configurion of a specific
 // type. The configType must be either "json" or the adapter type.
 func (tc *Tester) InitServer(rawConfig string, configType string) {
-
 	if err := tc.initServer(rawConfig, configType); err != nil {
 		tc.t.Logf("failed to load config: %s", err)
 		tc.t.Fail()
@@ -108,13 +129,12 @@ func (tc *Tester) InitServer(rawConfig string, configType string) {
 // InitServer this will configure the server with a configurion of a specific
 // type. The configType must be either "json" or the adapter type.
 func (tc *Tester) initServer(rawConfig string, configType string) error {
-
 	if testing.Short() {
 		tc.t.SkipNow()
 		return nil
 	}
 
-	err := validateTestPrerequisites()
+	err := validateTestPrerequisites(tc)
 	if err != nil {
 		tc.t.Skipf("skipping tests as failed integration prerequisites. %s", err)
 		return nil
@@ -122,8 +142,7 @@ func (tc *Tester) initServer(rawConfig string, configType string) error {
 
 	tc.t.Cleanup(func() {
 		if tc.t.Failed() && tc.configLoaded {
-
-			res, err := http.Get(fmt.Sprintf("http://localhost:%d/config/", Default.AdminPort))
+			res, err := http.Get(fmt.Sprintf("http://localhost:%d/config/", tc.config.AdminPort))
 			if err != nil {
 				tc.t.Log("unable to read the current config")
 				return
@@ -138,11 +157,25 @@ func (tc *Tester) initServer(rawConfig string, configType string) error {
 	})
 
 	rawConfig = prependCaddyFilePath(rawConfig)
+	// normalize JSON config
+	if configType == "json" {
+		tc.t.Logf("Before: %s", rawConfig)
+		var conf any
+		if err := json.Unmarshal([]byte(rawConfig), &conf); err != nil {
+			return err
+		}
+		c, err := json.Marshal(conf)
+		if err != nil {
+			return err
+		}
+		rawConfig = string(c)
+		tc.t.Logf("After: %s", rawConfig)
+	}
 	client := &http.Client{
-		Timeout: Default.LoadRequestTimeout,
+		Timeout: tc.config.LoadRequestTimeout,
 	}
 	start := time.Now()
-	req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/load", Default.AdminPort), strings.NewReader(rawConfig))
+	req, err := http.NewRequest("POST", fmt.Sprintf("http://localhost:%d/load", tc.config.AdminPort), strings.NewReader(rawConfig))
 	if err != nil {
 		tc.t.Errorf("failed to create request. %s", err)
 		return err
@@ -193,11 +226,11 @@ func (tc *Tester) ensureConfigRunning(rawConfig string, configType string) error
 	}
 
 	client := &http.Client{
-		Timeout: Default.LoadRequestTimeout,
+		Timeout: tc.config.LoadRequestTimeout,
 	}
 
 	fetchConfig := func(client *http.Client) any {
-		resp, err := client.Get(fmt.Sprintf("http://localhost:%d/config/", Default.AdminPort))
+		resp, err := client.Get(fmt.Sprintf("http://localhost:%d/config/", tc.config.AdminPort))
 		if err != nil {
 			return nil
 		}
@@ -214,52 +247,68 @@ func (tc *Tester) ensureConfigRunning(rawConfig string, configType string) error
 		return actual
 	}
 
-	for retries := 4; retries > 0; retries-- {
+	for retries := 10; retries > 0; retries-- {
 		if reflect.DeepEqual(expected, fetchConfig(client)) {
 			return nil
 		}
-		time.Sleep(10 * time.Millisecond)
+		time.Sleep(1 * time.Second)
 	}
 	tc.t.Errorf("POSTed configuration isn't active")
 	return errors.New("EnsureConfigRunning: POSTed configuration isn't active")
 }
 
+const initConfig = `{
+	admin localhost:%d
+}
+`
+
 // validateTestPrerequisites ensures the certificates are available in the
 // designated path and Caddy sub-process is running.
-func validateTestPrerequisites() error {
-
+func validateTestPrerequisites(tc *Tester) error {
 	// check certificates are found
-	for _, certName := range Default.Certifcates {
-		if _, err := os.Stat(getIntegrationDir() + certName); os.IsNotExist(err) {
+	for _, certName := range tc.config.Certificates {
+		if _, err := os.Stat(getIntegrationDir() + certName); errors.Is(err, fs.ErrNotExist) {
 			return fmt.Errorf("caddy integration test certificates (%s) not found", certName)
 		}
 	}
 
-	if isCaddyAdminRunning() != nil {
+	if isCaddyAdminRunning(tc) != nil {
+		// setup the init config file, and set the cleanup afterwards
+		f, err := os.CreateTemp("", "")
+		if err != nil {
+			return err
+		}
+		tc.t.Cleanup(func() {
+			os.Remove(f.Name())
+		})
+		if _, err := fmt.Fprintf(f, initConfig, tc.config.AdminPort); err != nil {
+			return err
+		}
+
 		// start inprocess caddy server
-		os.Args = []string{"caddy", "run"}
+		os.Args = []string{"caddy", "run", "--config", f.Name(), "--adapter", "caddyfile"}
 		go func() {
 			caddycmd.Main()
 		}()
 
 		// wait for caddy to start serving the initial config
-		for retries := 4; retries > 0 && isCaddyAdminRunning() != nil; retries-- {
-			time.Sleep(10 * time.Millisecond)
+		for retries := 10; retries > 0 && isCaddyAdminRunning(tc) != nil; retries-- {
+			time.Sleep(1 * time.Second)
 		}
 	}
 
 	// one more time to return the error
-	return isCaddyAdminRunning()
+	return isCaddyAdminRunning(tc)
 }
 
-func isCaddyAdminRunning() error {
+func isCaddyAdminRunning(tc *Tester) error {
 	// assert that caddy is running
 	client := &http.Client{
-		Timeout: Default.LoadRequestTimeout,
+		Timeout: tc.config.LoadRequestTimeout,
 	}
-	resp, err := client.Get(fmt.Sprintf("http://localhost:%d/config/", Default.AdminPort))
+	resp, err := client.Get(fmt.Sprintf("http://localhost:%d/config/", tc.config.AdminPort))
 	if err != nil {
-		return fmt.Errorf("caddy integration test caddy server not running. Expected to be listening on localhost:%d", Default.AdminPort)
+		return fmt.Errorf("caddy integration test caddy server not running. Expected to be listening on localhost:%d", tc.config.AdminPort)
 	}
 	resp.Body.Close()
 
@@ -267,7 +316,6 @@ func isCaddyAdminRunning() error {
 }
 
 func getIntegrationDir() string {
-
 	_, filename, _, ok := runtime.Caller(1)
 	if !ok {
 		panic("unable to determine the current file path")
@@ -287,7 +335,6 @@ func prependCaddyFilePath(rawConfig string) string {
 
 // CreateTestingTransport creates a testing transport that forces call dialing connections to happen locally
 func CreateTestingTransport() *http.Transport {
-
 	dialer := net.Dialer{
 		Timeout:   5 * time.Second,
 		KeepAlive: 5 * time.Second,
@@ -315,7 +362,6 @@ func CreateTestingTransport() *http.Transport {
 
 // AssertLoadError will load a config and expect an error
 func AssertLoadError(t *testing.T, rawConfig string, configType string, expectedError string) {
-
 	tc := NewTester(t)
 
 	err := tc.initServer(rawConfig, configType)
@@ -326,7 +372,6 @@ func AssertLoadError(t *testing.T, rawConfig string, configType string, expected
 
 // AssertRedirect makes a request and asserts the redirection happens
 func (tc *Tester) AssertRedirect(requestURI string, expectedToLocation string, expectedStatusCode int) *http.Response {
-
 	redirectPolicyFunc := func(req *http.Request, via []*http.Request) error {
 		return http.ErrUseLastResponse
 	}
@@ -363,8 +408,7 @@ func (tc *Tester) AssertRedirect(requestURI string, expectedToLocation string, e
 }
 
 // CompareAdapt adapts a config and then compares it against an expected result
-func CompareAdapt(t *testing.T, filename, rawConfig string, adapterName string, expectedResponse string) bool {
-
+func CompareAdapt(t testing.TB, filename, rawConfig string, adapterName string, expectedResponse string) bool {
 	cfgAdapter := caddyconfig.GetAdapter(adapterName)
 	if cfgAdapter == nil {
 		t.Logf("unrecognized config adapter '%s'", adapterName)
@@ -423,7 +467,7 @@ func CompareAdapt(t *testing.T, filename, rawConfig string, adapterName string,
 }
 
 // AssertAdapt adapts a config and then tests it against an expected result
-func AssertAdapt(t *testing.T, rawConfig string, adapterName string, expectedResponse string) {
+func AssertAdapt(t testing.TB, rawConfig string, adapterName string, expectedResponse string) {
 	ok := CompareAdapt(t, "Caddyfile", rawConfig, adapterName, expectedResponse)
 	if !ok {
 		t.Fail()
@@ -432,7 +476,7 @@ func AssertAdapt(t *testing.T, rawConfig string, adapterName string, expectedRes
 
 // Generic request functions
 
-func applyHeaders(t *testing.T, req *http.Request, requestHeaders []string) {
+func applyHeaders(t testing.TB, req *http.Request, requestHeaders []string) {
 	requestContentType := ""
 	for _, requestHeader := range requestHeaders {
 		arr := strings.SplitAfterN(requestHeader, ":", 2)
@@ -452,14 +496,13 @@ func applyHeaders(t *testing.T, req *http.Request, requestHeaders []string) {
 
 // AssertResponseCode will execute the request and verify the status code, returns a response for additional assertions
 func (tc *Tester) AssertResponseCode(req *http.Request, expectedStatusCode int) *http.Response {
-
 	resp, err := tc.Client.Do(req)
 	if err != nil {
 		tc.t.Fatalf("failed to call server %s", err)
 	}
 
 	if expectedStatusCode != resp.StatusCode {
-		tc.t.Errorf("requesting \"%s\" expected status code: %d but got %d", req.RequestURI, expectedStatusCode, resp.StatusCode)
+		tc.t.Errorf("requesting \"%s\" expected status code: %d but got %d", req.URL.RequestURI(), expectedStatusCode, resp.StatusCode)
 	}
 
 	return resp
@@ -467,7 +510,6 @@ func (tc *Tester) AssertResponseCode(req *http.Request, expectedStatusCode int)
 
 // AssertResponse request a URI and assert the status code and the body contains a string
 func (tc *Tester) AssertResponse(req *http.Request, expectedStatusCode int, expectedBody string) (*http.Response, string) {
-
 	resp := tc.AssertResponseCode(req, expectedStatusCode)
 
 	defer resp.Body.Close()
@@ -489,7 +531,6 @@ func (tc *Tester) AssertResponse(req *http.Request, expectedStatusCode int, expe
 
 // AssertGetResponse GET a URI and expect a statusCode and body text
 func (tc *Tester) AssertGetResponse(requestURI string, expectedStatusCode int, expectedBody string) (*http.Response, string) {
-
 	req, err := http.NewRequest("GET", requestURI, nil)
 	if err != nil {
 		tc.t.Fatalf("unable to create request %s", err)
@@ -500,7 +541,6 @@ func (tc *Tester) AssertGetResponse(requestURI string, expectedStatusCode int, e
 
 // AssertDeleteResponse request a URI and expect a statusCode and body text
 func (tc *Tester) AssertDeleteResponse(requestURI string, expectedStatusCode int, expectedBody string) (*http.Response, string) {
-
 	req, err := http.NewRequest("DELETE", requestURI, nil)
 	if err != nil {
 		tc.t.Fatalf("unable to create request %s", err)
@@ -511,7 +551,6 @@ func (tc *Tester) AssertDeleteResponse(requestURI string, expectedStatusCode int
 
 // AssertPostResponseBody POST to a URI and assert the response code and body
 func (tc *Tester) AssertPostResponseBody(requestURI string, requestHeaders []string, requestBody *bytes.Buffer, expectedStatusCode int, expectedBody string) (*http.Response, string) {
-
 	req, err := http.NewRequest("POST", requestURI, requestBody)
 	if err != nil {
 		tc.t.Errorf("failed to create request %s", err)
@@ -525,7 +564,6 @@ func (tc *Tester) AssertPostResponseBody(requestURI string, requestHeaders []str
 
 // AssertPutResponseBody PUT to a URI and assert the response code and body
 func (tc *Tester) AssertPutResponseBody(requestURI string, requestHeaders []string, requestBody *bytes.Buffer, expectedStatusCode int, expectedBody string) (*http.Response, string) {
-
 	req, err := http.NewRequest("PUT", requestURI, requestBody)
 	if err != nil {
 		tc.t.Errorf("failed to create request %s", err)
@@ -539,7 +577,6 @@ func (tc *Tester) AssertPutResponseBody(requestURI string, requestHeaders []stri
 
 // AssertPatchResponseBody PATCH to a URI and assert the response code and body
 func (tc *Tester) AssertPatchResponseBody(requestURI string, requestHeaders []string, requestBody *bytes.Buffer, expectedStatusCode int, expectedBody string) (*http.Response, string) {
-
 	req, err := http.NewRequest("PATCH", requestURI, requestBody)
 	if err != nil {
 		tc.t.Errorf("failed to create request %s", err)

caddytest/caddytest_test.go

@@ -1,6 +1,7 @@
 package caddytest
 
 import (
+	"net/http"
 	"strings"
 	"testing"
 )
@@ -31,3 +32,97 @@ func TestReplaceCertificatePaths(t *testing.T) {
 		t.Error("expected redirect uri to be unchanged")
 	}
 }
+
+func TestLoadUnorderedJSON(t *testing.T) {
+	tester := NewTester(t)
+	tester.InitServer(`
+	{
+		"logging": {
+			"logs": {
+				"default": {
+					"level": "DEBUG",
+					"writer": {
+						"output": "stdout"
+					}
+				},
+				"sStdOutLogs": {
+					"level": "DEBUG",
+					"writer": {
+						"output": "stdout"
+					},
+					"include": [
+						"http.*",
+						"admin.*"
+					]
+				},
+				"sFileLogs": {
+					"level": "DEBUG",
+					"writer": {
+						"output": "stdout"
+					},
+					"include": [
+						"http.*",
+						"admin.*"
+					]
+				}
+			}
+		},
+		"admin": {
+			"listen": "localhost:2999"
+		},
+		"apps": {
+			"pki": {
+				"certificate_authorities" : {
+				  "local" : {
+					"install_trust": false
+				  }
+				}
+			},
+			"http": {
+				"http_port": 9080,
+				"https_port": 9443,
+				"servers": {
+					"s_server": {
+						"listen": [
+							":9080"
+						],
+						"routes": [
+							{
+								"handle": [
+									{
+										"handler": "static_response",
+										"body": "Hello"
+									}
+								]
+							},
+							{
+								"match": [
+									{
+										"host": [
+											"localhost",
+											"127.0.0.1"
+										]
+									}
+								]
+							}
+						],
+						"logs": {
+							"default_logger_name": "sStdOutLogs",
+							"logger_names": {
+								"localhost": "sStdOutLogs",
+								"127.0.0.1": "sFileLogs"
+							}
+						}
+					}
+				}
+			}
+		}
+	}
+  `, "json")
+	req, err := http.NewRequest(http.MethodGet, "http://localhost:9080/", nil)
+	if err != nil {
+		t.Fail()
+		return
+	}
+	tester.AssertResponseCode(req, 200)
+}

caddytest/integration/acme_test.go

@@ -0,0 +1,209 @@
+package integration
+
+import (
+	"context"
+	"crypto/ecdsa"
+	"crypto/elliptic"
+	"crypto/rand"
+	"fmt"
+	"log/slog"
+	"net"
+	"net/http"
+	"strings"
+	"testing"
+
+	"github.com/mholt/acmez/v3"
+	"github.com/mholt/acmez/v3/acme"
+	smallstepacme "github.com/smallstep/certificates/acme"
+	"go.uber.org/zap"
+	"go.uber.org/zap/exp/zapslog"
+
+	"github.com/caddyserver/caddy/v2"
+	"github.com/caddyserver/caddy/v2/caddytest"
+)
+
+const acmeChallengePort = 9081
+
+// Test the basic functionality of Caddy's ACME server
+func TestACMEServerWithDefaults(t *testing.T) {
+	ctx := context.Background()
+	logger, err := zap.NewDevelopment()
+	if err != nil {
+		t.Error(err)
+		return
+	}
+
+	tester := caddytest.NewTester(t)
+	tester.InitServer(`
+	{
+		skip_install_trust
+		admin localhost:2999
+		http_port     9080
+		https_port    9443
+		local_certs
+	}
+	acme.localhost {
+		acme_server
+	}
+  `, "caddyfile")
+
+	client := acmez.Client{
+		Client: &acme.Client{
+			Directory:  "https://acme.localhost:9443/acme/local/directory",
+			HTTPClient: tester.Client,
+			Logger:     slog.New(zapslog.NewHandler(logger.Core())),
+		},
+		ChallengeSolvers: map[string]acmez.Solver{
+			acme.ChallengeTypeHTTP01: &naiveHTTPSolver{logger: logger},
+		},
+	}
+
+	accountPrivateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Errorf("generating account key: %v", err)
+	}
+	account := acme.Account{
+		Contact:              []string{"mailto:you@example.com"},
+		TermsOfServiceAgreed: true,
+		PrivateKey:           accountPrivateKey,
+	}
+	account, err = client.NewAccount(ctx, account)
+	if err != nil {
+		t.Errorf("new account: %v", err)
+		return
+	}
+
+	// Every certificate needs a key.
+	certPrivateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Errorf("generating certificate key: %v", err)
+		return
+	}
+
+	certs, err := client.ObtainCertificateForSANs(ctx, account, certPrivateKey, []string{"localhost"})
+	if err != nil {
+		t.Errorf("obtaining certificate: %v", err)
+		return
+	}
+
+	// ACME servers should usually give you the entire certificate chain
+	// in PEM format, and sometimes even alternate chains! It's up to you
+	// which one(s) to store and use, but whatever you do, be sure to
+	// store the certificate and key somewhere safe and secure, i.e. don't
+	// lose them!
+	for _, cert := range certs {
+		t.Logf("Certificate %q:\n%s\n\n", cert.URL, cert.ChainPEM)
+	}
+}
+
+func TestACMEServerWithMismatchedChallenges(t *testing.T) {
+	ctx := context.Background()
+	logger := caddy.Log().Named("acmez")
+
+	tester := caddytest.NewTester(t)
+	tester.InitServer(`
+	{
+		skip_install_trust
+		admin localhost:2999
+		http_port     9080
+		https_port    9443
+		local_certs
+	}
+	acme.localhost {
+		acme_server {
+			challenges tls-alpn-01
+		}
+	}
+  `, "caddyfile")
+
+	client := acmez.Client{
+		Client: &acme.Client{
+			Directory:  "https://acme.localhost:9443/acme/local/directory",
+			HTTPClient: tester.Client,
+			Logger:     slog.New(zapslog.NewHandler(logger.Core())),
+		},
+		ChallengeSolvers: map[string]acmez.Solver{
+			acme.ChallengeTypeHTTP01: &naiveHTTPSolver{logger: logger},
+		},
+	}
+
+	accountPrivateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Errorf("generating account key: %v", err)
+	}
+	account := acme.Account{
+		Contact:              []string{"mailto:you@example.com"},
+		TermsOfServiceAgreed: true,
+		PrivateKey:           accountPrivateKey,
+	}
+	account, err = client.NewAccount(ctx, account)
+	if err != nil {
+		t.Errorf("new account: %v", err)
+		return
+	}
+
+	// Every certificate needs a key.
+	certPrivateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Errorf("generating certificate key: %v", err)
+		return
+	}
+
+	certs, err := client.ObtainCertificateForSANs(ctx, account, certPrivateKey, []string{"localhost"})
+	if len(certs) > 0 {
+		t.Errorf("expected '0' certificates, but received '%d'", len(certs))
+	}
+	if err == nil {
+		t.Error("expected errors, but received none")
+	}
+	const expectedErrMsg = "no solvers available for remaining challenges (configured=[http-01] offered=[tls-alpn-01] remaining=[tls-alpn-01])"
+	if !strings.Contains(err.Error(), expectedErrMsg) {
+		t.Errorf(`received error message does not match expectation: expected="%s" received="%s"`, expectedErrMsg, err.Error())
+	}
+}
+
+// naiveHTTPSolver is a no-op acmez.Solver for example purposes only.
+type naiveHTTPSolver struct {
+	srv    *http.Server
+	logger *zap.Logger
+}
+
+func (s *naiveHTTPSolver) Present(ctx context.Context, challenge acme.Challenge) error {
+	smallstepacme.InsecurePortHTTP01 = acmeChallengePort
+	s.srv = &http.Server{
+		Addr: fmt.Sprintf(":%d", acmeChallengePort),
+		Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			host, _, err := net.SplitHostPort(r.Host)
+			if err != nil {
+				host = r.Host
+			}
+			s.logger.Info("received request on challenge server", zap.String("path", r.URL.Path))
+			if r.Method == "GET" && r.URL.Path == challenge.HTTP01ResourcePath() && strings.EqualFold(host, challenge.Identifier.Value) {
+				w.Header().Add("Content-Type", "text/plain")
+				w.Write([]byte(challenge.KeyAuthorization))
+				r.Close = true
+				s.logger.Info("served key authentication",
+					zap.String("identifier", challenge.Identifier.Value),
+					zap.String("challenge", "http-01"),
+					zap.String("remote", r.RemoteAddr),
+				)
+			}
+		}),
+	}
+	l, err := net.Listen("tcp", fmt.Sprintf(":%d", acmeChallengePort))
+	if err != nil {
+		return err
+	}
+	s.logger.Info("present challenge", zap.Any("challenge", challenge))
+	go s.srv.Serve(l)
+	return nil
+}
+
+func (s naiveHTTPSolver) CleanUp(ctx context.Context, challenge acme.Challenge) error {
+	smallstepacme.InsecurePortHTTP01 = 0
+	s.logger.Info("cleanup", zap.Any("challenge", challenge))
+	if s.srv != nil {
+		s.srv.Close()
+	}
+	return nil
+}

caddytest/integration/acmeserver_test.go

@@ -0,0 +1,207 @@
+package integration
+
+import (
+	"context"
+	"crypto/ecdsa"
+	"crypto/elliptic"
+	"crypto/rand"
+	"log/slog"
+	"strings"
+	"testing"
+
+	"github.com/mholt/acmez/v3"
+	"github.com/mholt/acmez/v3/acme"
+	"go.uber.org/zap"
+	"go.uber.org/zap/exp/zapslog"
+
+	"github.com/caddyserver/caddy/v2/caddytest"
+)
+
+func TestACMEServerDirectory(t *testing.T) {
+	tester := caddytest.NewTester(t)
+	tester.InitServer(`
+	{
+		skip_install_trust
+		local_certs
+		admin localhost:2999
+		http_port     9080
+		https_port    9443
+		pki {
+			ca local {
+				name "Caddy Local Authority"
+			}
+		}
+	}
+	acme.localhost:9443 {
+		acme_server
+	}
+  `, "caddyfile")
+	tester.AssertGetResponse(
+		"https://acme.localhost:9443/acme/local/directory",
+		200,
+		`{"newNonce":"https://acme.localhost:9443/acme/local/new-nonce","newAccount":"https://acme.localhost:9443/acme/local/new-account","newOrder":"https://acme.localhost:9443/acme/local/new-order","revokeCert":"https://acme.localhost:9443/acme/local/revoke-cert","keyChange":"https://acme.localhost:9443/acme/local/key-change"}
+`)
+}
+
+func TestACMEServerAllowPolicy(t *testing.T) {
+	tester := caddytest.NewTester(t)
+	tester.InitServer(`
+	{
+		skip_install_trust
+		local_certs
+		admin localhost:2999
+		http_port     9080
+		https_port    9443
+		pki {
+			ca local {
+				name "Caddy Local Authority"
+			}
+		}
+	}
+	acme.localhost {
+		acme_server {
+			challenges http-01
+			allow {
+				domains localhost
+			}
+		}
+	}
+  `, "caddyfile")
+
+	ctx := context.Background()
+	logger, err := zap.NewDevelopment()
+	if err != nil {
+		t.Error(err)
+		return
+	}
+
+	client := acmez.Client{
+		Client: &acme.Client{
+			Directory:  "https://acme.localhost:9443/acme/local/directory",
+			HTTPClient: tester.Client,
+			Logger:     slog.New(zapslog.NewHandler(logger.Core())),
+		},
+		ChallengeSolvers: map[string]acmez.Solver{
+			acme.ChallengeTypeHTTP01: &naiveHTTPSolver{logger: logger},
+		},
+	}
+
+	accountPrivateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Errorf("generating account key: %v", err)
+	}
+	account := acme.Account{
+		Contact:              []string{"mailto:you@example.com"},
+		TermsOfServiceAgreed: true,
+		PrivateKey:           accountPrivateKey,
+	}
+	account, err = client.NewAccount(ctx, account)
+	if err != nil {
+		t.Errorf("new account: %v", err)
+		return
+	}
+
+	// Every certificate needs a key.
+	certPrivateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Errorf("generating certificate key: %v", err)
+		return
+	}
+	{
+		certs, err := client.ObtainCertificateForSANs(ctx, account, certPrivateKey, []string{"localhost"})
+		if err != nil {
+			t.Errorf("obtaining certificate for allowed domain: %v", err)
+			return
+		}
+
+		// ACME servers should usually give you the entire certificate chain
+		// in PEM format, and sometimes even alternate chains! It's up to you
+		// which one(s) to store and use, but whatever you do, be sure to
+		// store the certificate and key somewhere safe and secure, i.e. don't
+		// lose them!
+		for _, cert := range certs {
+			t.Logf("Certificate %q:\n%s\n\n", cert.URL, cert.ChainPEM)
+		}
+	}
+	{
+		_, err := client.ObtainCertificateForSANs(ctx, account, certPrivateKey, []string{"not-matching.localhost"})
+		if err == nil {
+			t.Errorf("obtaining certificate for 'not-matching.localhost' domain")
+		} else if err != nil && !strings.Contains(err.Error(), "urn:ietf:params:acme:error:rejectedIdentifier") {
+			t.Logf("unexpected error: %v", err)
+		}
+	}
+}
+
+func TestACMEServerDenyPolicy(t *testing.T) {
+	tester := caddytest.NewTester(t)
+	tester.InitServer(`
+	{
+		skip_install_trust
+		local_certs
+		admin localhost:2999
+		http_port     9080
+		https_port    9443
+		pki {
+			ca local {
+				name "Caddy Local Authority"
+			}
+		}
+	}
+	acme.localhost {
+		acme_server {
+			deny {
+				domains deny.localhost
+			}
+		}
+	}
+  `, "caddyfile")
+
+	ctx := context.Background()
+	logger, err := zap.NewDevelopment()
+	if err != nil {
+		t.Error(err)
+		return
+	}
+
+	client := acmez.Client{
+		Client: &acme.Client{
+			Directory:  "https://acme.localhost:9443/acme/local/directory",
+			HTTPClient: tester.Client,
+			Logger:     slog.New(zapslog.NewHandler(logger.Core())),
+		},
+		ChallengeSolvers: map[string]acmez.Solver{
+			acme.ChallengeTypeHTTP01: &naiveHTTPSolver{logger: logger},
+		},
+	}
+
+	accountPrivateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Errorf("generating account key: %v", err)
+	}
+	account := acme.Account{
+		Contact:              []string{"mailto:you@example.com"},
+		TermsOfServiceAgreed: true,
+		PrivateKey:           accountPrivateKey,
+	}
+	account, err = client.NewAccount(ctx, account)
+	if err != nil {
+		t.Errorf("new account: %v", err)
+		return
+	}
+
+	// Every certificate needs a key.
+	certPrivateKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
+	if err != nil {
+		t.Errorf("generating certificate key: %v", err)
+		return
+	}
+	{
+		_, err := client.ObtainCertificateForSANs(ctx, account, certPrivateKey, []string{"deny.localhost"})
+		if err == nil {
+			t.Errorf("obtaining certificate for 'deny.localhost' domain")
+		} else if err != nil && !strings.Contains(err.Error(), "urn:ietf:params:acme:error:rejectedIdentifier") {
+			t.Logf("unexpected error: %v", err)
+		}
+	}
+}

caddytest/integration/autohttps_test.go

@@ -11,6 +11,8 @@ func TestAutoHTTPtoHTTPSRedirectsImplicitPort(t *testing.T) {
 	tester := caddytest.NewTester(t)
 	tester.InitServer(`
 	{
+		admin localhost:2999
+		skip_install_trust
 		http_port     9080
 		https_port    9443
 	}
@@ -25,6 +27,8 @@ func TestAutoHTTPtoHTTPSRedirectsExplicitPortSameAsHTTPSPort(t *testing.T) {
 	tester := caddytest.NewTester(t)
 	tester.InitServer(`
 	{
+		skip_install_trust
+		admin localhost:2999
 		http_port     9080
 		https_port    9443
 	}
@@ -39,6 +43,8 @@ func TestAutoHTTPtoHTTPSRedirectsExplicitPortDifferentFromHTTPSPort(t *testing.T
 	tester := caddytest.NewTester(t)
 	tester.InitServer(`
 	{
+		skip_install_trust
+		admin localhost:2999
 		http_port     9080
 		https_port    9443
 	}
@@ -53,6 +59,9 @@ func TestAutoHTTPRedirectsWithHTTPListenerFirstInAddresses(t *testing.T) {
 	tester := caddytest.NewTester(t)
 	tester.InitServer(`
 {
+  "admin": {
+	"listen": "localhost:2999"
+  },
   "apps": {
     "http": {
       "http_port": 9080,
@@ -74,7 +83,14 @@ func TestAutoHTTPRedirectsWithHTTPListenerFirstInAddresses(t *testing.T) {
           ]
         }
       }
-    }
+    },
+	"pki": {
+		"certificate_authorities": {
+			"local": {
+				"install_trust": false
+			}
+		}
+	}
   }
 }
 `, "json")
@@ -85,6 +101,8 @@ func TestAutoHTTPRedirectsInsertedBeforeUserDefinedCatchAll(t *testing.T) {
 	tester := caddytest.NewTester(t)
 	tester.InitServer(`
 	{
+		skip_install_trust
+		admin localhost:2999
 		http_port     9080
 		https_port    9443
 		local_certs
@@ -108,6 +126,8 @@ func TestAutoHTTPRedirectsInsertedBeforeUserDefinedCatchAllWithNoExplicitHTTPSit
 	tester := caddytest.NewTester(t)
 	tester.InitServer(`
 	{
+		skip_install_trust
+		admin localhost:2999
 		http_port     9080
 		https_port    9443
 		local_certs

caddytest/integration/caddyfile_adapt/acme_dns_naked_use_dns_defaults.caddyfiletest

@@ -0,0 +1,60 @@
+{
+	dns mock
+	acme_dns
+}
+
+example.com {
+
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":443"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"example.com"
+									]
+								}
+							],
+							"terminal": true
+						}
+					]
+				}
+			}
+		},
+		"tls": {
+			"automation": {
+				"policies": [
+					{
+						"subjects": [
+							"example.com"
+						],
+						"issuers": [
+							{
+								"challenges": {
+									"dns": {
+										"provider": {
+											"name": "mock"
+										}
+									}
+								},
+								"module": "acme"
+							}
+						]
+					}
+				]
+			},
+			"dns": {
+				"name": "mock"
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/acme_server_custom_challenges.caddyfiletest

@@ -0,0 +1,65 @@
+{
+	pki {
+		ca custom-ca {
+			name "Custom CA"
+		}
+	}
+}
+
+acme.example.com {
+	acme_server {
+		ca custom-ca
+		challenges dns-01
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":443"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"acme.example.com"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"ca": "custom-ca",
+													"challenges": [
+														"dns-01"
+													],
+													"handler": "acme_server"
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					]
+				}
+			}
+		},
+		"pki": {
+			"certificate_authorities": {
+				"custom-ca": {
+					"name": "Custom CA"
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/acme_server_default_challenges.caddyfiletest

@@ -0,0 +1,62 @@
+{
+	pki {
+		ca custom-ca {
+			name "Custom CA"
+		}
+	}
+}
+
+acme.example.com {
+	acme_server {
+		ca custom-ca
+		challenges
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":443"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"acme.example.com"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"ca": "custom-ca",
+													"handler": "acme_server"
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					]
+				}
+			}
+		},
+		"pki": {
+			"certificate_authorities": {
+				"custom-ca": {
+					"name": "Custom CA"
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/acme_server_lifetime.caddyfiletest

@@ -0,0 +1,108 @@
+{
+	pki {
+		ca internal {
+			name "Internal"
+			root_cn "Internal Root Cert"
+			intermediate_cn "Internal Intermediate Cert"
+		}
+		ca internal-long-lived {
+			name "Long-lived"
+			root_cn "Internal Root Cert 2"
+			intermediate_cn "Internal Intermediate Cert 2"
+		}
+	}
+}
+
+acme-internal.example.com {
+	acme_server {
+		ca internal
+	}
+}
+
+acme-long-lived.example.com {
+	acme_server {
+		ca internal-long-lived
+		lifetime 7d
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":443"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"acme-long-lived.example.com"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"ca": "internal-long-lived",
+													"handler": "acme_server",
+													"lifetime": 604800000000000
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						},
+						{
+							"match": [
+								{
+									"host": [
+										"acme-internal.example.com"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"ca": "internal",
+													"handler": "acme_server"
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					]
+				}
+			}
+		},
+		"pki": {
+			"certificate_authorities": {
+				"internal": {
+					"name": "Internal",
+					"root_common_name": "Internal Root Cert",
+					"intermediate_common_name": "Internal Intermediate Cert"
+				},
+				"internal-long-lived": {
+					"name": "Long-lived",
+					"root_common_name": "Internal Root Cert 2",
+					"intermediate_common_name": "Internal Intermediate Cert 2"
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/acme_server_multi_custom_challenges.caddyfiletest

@@ -0,0 +1,66 @@
+{
+	pki {
+		ca custom-ca {
+			name "Custom CA"
+		}
+	}
+}
+
+acme.example.com {
+	acme_server {
+		ca custom-ca
+		challenges dns-01 http-01
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":443"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"acme.example.com"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"ca": "custom-ca",
+													"challenges": [
+														"dns-01",
+														"http-01"
+													],
+													"handler": "acme_server"
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					]
+				}
+			}
+		},
+		"pki": {
+			"certificate_authorities": {
+				"custom-ca": {
+					"name": "Custom CA"
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/acme_server_policy-allow.caddyfiletest

@@ -0,0 +1,72 @@
+{
+	pki {
+		ca custom-ca {
+			name "Custom CA"
+		}
+	}
+}
+
+acme.example.com {
+	acme_server {
+		ca custom-ca
+		allow {
+			domains host-1.internal.example.com host-2.internal.example.com
+		}
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":443"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"acme.example.com"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"ca": "custom-ca",
+													"handler": "acme_server",
+													"policy": {
+														"allow": {
+															"domains": [
+																"host-1.internal.example.com",
+																"host-2.internal.example.com"
+															]
+														}
+													}
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					]
+				}
+			}
+		},
+		"pki": {
+			"certificate_authorities": {
+				"custom-ca": {
+					"name": "Custom CA"
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/acme_server_policy-both.caddyfiletest

@@ -0,0 +1,80 @@
+{
+	pki {
+		ca custom-ca {
+			name "Custom CA"
+		}
+	}
+}
+
+acme.example.com {
+	acme_server {
+		ca custom-ca
+		allow {
+			domains host-1.internal.example.com host-2.internal.example.com
+		}
+		deny {
+			domains dc.internal.example.com
+		}
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":443"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"acme.example.com"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"ca": "custom-ca",
+													"handler": "acme_server",
+													"policy": {
+														"allow": {
+															"domains": [
+																"host-1.internal.example.com",
+																"host-2.internal.example.com"
+															]
+														},
+														"deny": {
+															"domains": [
+																"dc.internal.example.com"
+															]
+														}
+													}
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					]
+				}
+			}
+		},
+		"pki": {
+			"certificate_authorities": {
+				"custom-ca": {
+					"name": "Custom CA"
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/acme_server_policy-deny.caddyfiletest

@@ -0,0 +1,71 @@
+{
+	pki {
+		ca custom-ca {
+			name "Custom CA"
+		}
+	}
+}
+
+acme.example.com {
+	acme_server {
+		ca custom-ca
+		deny {
+			domains dc.internal.example.com
+		}
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":443"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"acme.example.com"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"ca": "custom-ca",
+													"handler": "acme_server",
+													"policy": {
+														"deny": {
+															"domains": [
+																"dc.internal.example.com"
+															]
+														}
+													}
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					]
+				}
+			}
+		},
+		"pki": {
+			"certificate_authorities": {
+				"custom-ca": {
+					"name": "Custom CA"
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/acme_server_sign_with_root.caddyfiletest

@@ -0,0 +1,67 @@
+{
+	pki {
+		ca internal {
+			name "Internal"
+			root_cn "Internal Root Cert"
+			intermediate_cn "Internal Intermediate Cert"
+		}
+	}
+}
+acme.example.com {
+	acme_server {
+		ca internal
+		sign_with_root
+	}
+}
+
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":443"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"acme.example.com"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"ca": "internal",
+													"handler": "acme_server",
+													"sign_with_root": true
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					]
+				}
+			}
+		},
+		"pki": {
+			"certificate_authorities": {
+				"internal": {
+					"name": "Internal",
+					"root_common_name": "Internal Root Cert",
+					"intermediate_common_name": "Internal Intermediate Cert"
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/ambiguous_site_definition.caddyfiletest

@@ -0,0 +1,12 @@
+example.com
+handle {
+	respond "one"
+}
+
+example.com
+handle {
+	respond "two"
+}
+----------
+Caddyfile:6: unrecognized directive: example.com
+Did you mean to define a second site? If so, you must use curly braces around each site to separate their configurations.

caddytest/integration/caddyfile_adapt/ambiguous_site_definition_duplicate_key.caddyfiletest

@@ -0,0 +1,9 @@
+:8080 {
+	respond "one"
+}
+
+:8080 {
+	respond "two"
+}
+----------
+ambiguous site definition: :8080

caddytest/integration/caddyfile_adapt/bind_fd_fdgram_h123.caddyfiletest

@@ -0,0 +1,142 @@
+{
+	auto_https disable_redirects
+	admin off
+}
+
+http://localhost {
+	bind fd/{env.CADDY_HTTP_FD} {
+		protocols h1
+	}
+	log
+	respond "Hello, HTTP!"
+}
+
+https://localhost {
+	bind fd/{env.CADDY_HTTPS_FD} {
+		protocols h1 h2
+	}
+	bind fdgram/{env.CADDY_HTTP3_FD} {
+		protocols h3
+	}
+	log
+	respond "Hello, HTTPS!"
+}
+----------
+{
+	"admin": {
+		"disabled": true
+	},
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						"fd/{env.CADDY_HTTPS_FD}",
+						"fdgram/{env.CADDY_HTTP3_FD}"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"localhost"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"body": "Hello, HTTPS!",
+													"handler": "static_response"
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					],
+					"automatic_https": {
+						"disable_redirects": true
+					},
+					"logs": {
+						"logger_names": {
+							"localhost": [
+								""
+							]
+						}
+					},
+					"listen_protocols": [
+						[
+							"h1",
+							"h2"
+						],
+						[
+							"h3"
+						]
+					]
+				},
+				"srv1": {
+					"automatic_https": {
+						"disable_redirects": true
+					}
+				},
+				"srv2": {
+					"listen": [
+						"fd/{env.CADDY_HTTP_FD}"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"localhost"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"body": "Hello, HTTP!",
+													"handler": "static_response"
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					],
+					"automatic_https": {
+						"disable_redirects": true,
+						"skip": [
+							"localhost"
+						]
+					},
+					"logs": {
+						"logger_names": {
+							"localhost": [
+								""
+							]
+						}
+					},
+					"listen_protocols": [
+						[
+							"h1"
+						]
+					]
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/directive_as_site_address.caddyfiletest

@@ -0,0 +1,5 @@
+handle
+
+respond "should not work"
+----------
+Caddyfile:1: parsed 'handle' as a site address, but it is a known directive; directives must appear in a site block

caddytest/integration/caddyfile_adapt/duplicate_listener_address_global.caddyfiletest

@@ -0,0 +1,12 @@
+{
+	servers {
+		srv0 {
+			listen :8080
+		}
+		srv1 {
+			listen :8080
+		}
+	}
+}
+----------
+parsing caddyfile tokens for 'servers': unrecognized servers option 'srv0', at Caddyfile:3

caddytest/integration/caddyfile_adapt/enable_tls_for_catch_all_site.caddyfiletest

@@ -0,0 +1,37 @@
+:8443 {
+	tls internal {
+		on_demand
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":8443"
+					],
+					"tls_connection_policies": [
+						{}
+					]
+				}
+			}
+		},
+		"tls": {
+			"automation": {
+				"policies": [
+					{
+						"issuers": [
+							{
+								"module": "internal"
+							}
+						],
+						"on_demand": true
+					}
+				]
+			}
+		}
+	}
+}
+

caddytest/integration/caddyfile_adapt/encode_options.caddyfiletest

@@ -11,6 +11,7 @@ encode gzip zstd {
 		header Content-Type application/xhtml+xml*
 		header Content-Type application/atom+xml*
 		header Content-Type application/rss+xml*
+		header Content-Type application/wasm*
 		header Content-Type image/svg+xml*
 	}
 }
@@ -20,6 +21,8 @@ encode {
 	zstd
 	gzip 5
 }
+
+encode
 ----------
 {
 	"apps": {
@@ -47,6 +50,7 @@ encode {
 												"application/xhtml+xml*",
 												"application/atom+xml*",
 												"application/rss+xml*",
+												"application/wasm*",
 												"image/svg+xml*"
 											]
 										},
@@ -74,6 +78,17 @@ encode {
 										"zstd",
 										"gzip"
 									]
+								},
+								{
+									"encodings": {
+										"gzip": {},
+										"zstd": {}
+									},
+									"handler": "encode",
+									"prefer": [
+										"zstd",
+										"gzip"
+									]
 								}
 							]
 						}

caddytest/integration/caddyfile_adapt/error_example.caddyfiletest

@@ -106,20 +106,29 @@ example.com {
 										"handler": "subroute",
 										"routes": [
 											{
-												"group": "group0",
 												"handle": [
 													{
-														"handler": "rewrite",
-														"uri": "/{http.error.status_code}.html"
-													}
-												]
-											},
-											{
-												"handle": [
-													{
-														"handler": "file_server",
-														"hide": [
-															"./Caddyfile"
+														"handler": "subroute",
+														"routes": [
+															{
+																"group": "group0",
+																"handle": [
+																	{
+																		"handler": "rewrite",
+																		"uri": "/{http.error.status_code}.html"
+																	}
+																]
+															},
+															{
+																"handle": [
+																	{
+																		"handler": "file_server",
+																		"hide": [
+																			"./Caddyfile"
+																		]
+																	}
+																]
+															}
 														]
 													}
 												]

caddytest/integration/caddyfile_adapt/error_multi_site_blocks.caddyfiletest

@@ -0,0 +1,281 @@
+foo.localhost {
+	root * /srv
+	error /private* "Unauthorized" 410
+	error /fivehundred* "Internal Server Error" 500
+
+	handle_errors 5xx {
+		respond "Error In range [500 .. 599]"
+	}
+	handle_errors 410 {
+		respond "404 or 410 error"
+	}
+}
+
+bar.localhost {
+	root * /srv
+	error /private* "Unauthorized" 410
+	error /fivehundred* "Internal Server Error" 500
+
+	handle_errors 5xx {
+		respond "Error In range [500 .. 599] from second site"
+	}
+	handle_errors 410 {
+		respond "404 or 410 error from second site"
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":443"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"foo.localhost"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"handler": "vars",
+													"root": "/srv"
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Internal Server Error",
+													"handler": "error",
+													"status_code": 500
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/fivehundred*"
+													]
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Unauthorized",
+													"handler": "error",
+													"status_code": 410
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/private*"
+													]
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						},
+						{
+							"match": [
+								{
+									"host": [
+										"bar.localhost"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"handler": "vars",
+													"root": "/srv"
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Internal Server Error",
+													"handler": "error",
+													"status_code": 500
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/fivehundred*"
+													]
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Unauthorized",
+													"handler": "error",
+													"status_code": 410
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/private*"
+													]
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					],
+					"errors": {
+						"routes": [
+							{
+								"match": [
+									{
+										"host": [
+											"foo.localhost"
+										]
+									}
+								],
+								"handle": [
+									{
+										"handler": "subroute",
+										"routes": [
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"handle": [
+																	{
+																		"body": "404 or 410 error",
+																		"handler": "static_response"
+																	}
+																]
+															}
+														]
+													}
+												],
+												"match": [
+													{
+														"expression": "{http.error.status_code} in [410]"
+													}
+												]
+											},
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"handle": [
+																	{
+																		"body": "Error In range [500 .. 599]",
+																		"handler": "static_response"
+																	}
+																]
+															}
+														]
+													}
+												],
+												"match": [
+													{
+														"expression": "{http.error.status_code} \u003e= 500 \u0026\u0026 {http.error.status_code} \u003c= 599"
+													}
+												]
+											}
+										]
+									}
+								],
+								"terminal": true
+							},
+							{
+								"match": [
+									{
+										"host": [
+											"bar.localhost"
+										]
+									}
+								],
+								"handle": [
+									{
+										"handler": "subroute",
+										"routes": [
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"handle": [
+																	{
+																		"body": "404 or 410 error from second site",
+																		"handler": "static_response"
+																	}
+																]
+															}
+														]
+													}
+												],
+												"match": [
+													{
+														"expression": "{http.error.status_code} in [410]"
+													}
+												]
+											},
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"handle": [
+																	{
+																		"body": "Error In range [500 .. 599] from second site",
+																		"handler": "static_response"
+																	}
+																]
+															}
+														]
+													}
+												],
+												"match": [
+													{
+														"expression": "{http.error.status_code} \u003e= 500 \u0026\u0026 {http.error.status_code} \u003c= 599"
+													}
+												]
+											}
+										]
+									}
+								],
+								"terminal": true
+							}
+						]
+					}
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/error_range_codes.caddyfiletest

@@ -0,0 +1,129 @@
+{
+	http_port 3010
+}
+localhost:3010 {
+	root * /srv
+	error /private* "Unauthorized" 410
+	error /hidden* "Not found" 404
+
+	handle_errors 4xx {
+		respond "Error in the [400 .. 499] range"
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"http_port": 3010,
+			"servers": {
+				"srv0": {
+					"listen": [
+						":3010"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"localhost"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"handler": "vars",
+													"root": "/srv"
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Unauthorized",
+													"handler": "error",
+													"status_code": 410
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/private*"
+													]
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Not found",
+													"handler": "error",
+													"status_code": 404
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/hidden*"
+													]
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					],
+					"errors": {
+						"routes": [
+							{
+								"match": [
+									{
+										"host": [
+											"localhost"
+										]
+									}
+								],
+								"handle": [
+									{
+										"handler": "subroute",
+										"routes": [
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"handle": [
+																	{
+																		"body": "Error in the [400 .. 499] range",
+																		"handler": "static_response"
+																	}
+																]
+															}
+														]
+													}
+												],
+												"match": [
+													{
+														"expression": "{http.error.status_code} \u003e= 400 \u0026\u0026 {http.error.status_code} \u003c= 499"
+													}
+												]
+											}
+										]
+									}
+								],
+								"terminal": true
+							}
+						]
+					}
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/error_range_simple_codes.caddyfiletest

@@ -0,0 +1,171 @@
+{
+	http_port 2099
+}
+localhost:2099 {
+	root * /srv
+	error /private* "Unauthorized" 410
+	error /threehundred* "Moved Permanently" 301
+	error /internalerr* "Internal Server Error" 500
+
+	handle_errors 500 3xx {
+		respond "Error code is equal to 500 or in the [300..399] range"
+	}
+	handle_errors 4xx {
+		respond "Error in the [400 .. 499] range"
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"http_port": 2099,
+			"servers": {
+				"srv0": {
+					"listen": [
+						":2099"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"localhost"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"handler": "vars",
+													"root": "/srv"
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Moved Permanently",
+													"handler": "error",
+													"status_code": 301
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/threehundred*"
+													]
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Internal Server Error",
+													"handler": "error",
+													"status_code": 500
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/internalerr*"
+													]
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Unauthorized",
+													"handler": "error",
+													"status_code": 410
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/private*"
+													]
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					],
+					"errors": {
+						"routes": [
+							{
+								"match": [
+									{
+										"host": [
+											"localhost"
+										]
+									}
+								],
+								"handle": [
+									{
+										"handler": "subroute",
+										"routes": [
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"handle": [
+																	{
+																		"body": "Error in the [400 .. 499] range",
+																		"handler": "static_response"
+																	}
+																]
+															}
+														]
+													}
+												],
+												"match": [
+													{
+														"expression": "{http.error.status_code} \u003e= 400 \u0026\u0026 {http.error.status_code} \u003c= 499"
+													}
+												]
+											},
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"handle": [
+																	{
+																		"body": "Error code is equal to 500 or in the [300..399] range",
+																		"handler": "static_response"
+																	}
+																]
+															}
+														]
+													}
+												],
+												"match": [
+													{
+														"expression": "{http.error.status_code} \u003e= 300 \u0026\u0026 {http.error.status_code} \u003c= 399 || {http.error.status_code} in [500]"
+													}
+												]
+											}
+										]
+									}
+								],
+								"terminal": true
+							}
+						]
+					}
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/error_simple_codes.caddyfiletest

@@ -0,0 +1,129 @@
+{
+	http_port 3010
+}
+localhost:3010 {
+	root * /srv
+	error /private* "Unauthorized" 410
+	error /hidden* "Not found" 404
+
+	handle_errors 404 410 {
+		respond "404 or 410 error"
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"http_port": 3010,
+			"servers": {
+				"srv0": {
+					"listen": [
+						":3010"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"localhost"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"handler": "vars",
+													"root": "/srv"
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Unauthorized",
+													"handler": "error",
+													"status_code": 410
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/private*"
+													]
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Not found",
+													"handler": "error",
+													"status_code": 404
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/hidden*"
+													]
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					],
+					"errors": {
+						"routes": [
+							{
+								"match": [
+									{
+										"host": [
+											"localhost"
+										]
+									}
+								],
+								"handle": [
+									{
+										"handler": "subroute",
+										"routes": [
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"handle": [
+																	{
+																		"body": "404 or 410 error",
+																		"handler": "static_response"
+																	}
+																]
+															}
+														]
+													}
+												],
+												"match": [
+													{
+														"expression": "{http.error.status_code} in [404, 410]"
+													}
+												]
+											}
+										]
+									}
+								],
+								"terminal": true
+							}
+						]
+					}
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/error_sort.caddyfiletest

@@ -0,0 +1,166 @@
+{
+	http_port 2099
+}
+localhost:2099 {
+	root * /srv
+	error /private* "Unauthorized" 410
+	error /hidden* "Not found" 404
+	error /internalerr* "Internal Server Error" 500
+
+	handle_errors {
+		respond "Fallback route: code outside the [400..499] range"
+	}
+	handle_errors 4xx {
+		respond "Error in the [400 .. 499] range"
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"http_port": 2099,
+			"servers": {
+				"srv0": {
+					"listen": [
+						":2099"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"localhost"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"handler": "vars",
+													"root": "/srv"
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Internal Server Error",
+													"handler": "error",
+													"status_code": 500
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/internalerr*"
+													]
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Unauthorized",
+													"handler": "error",
+													"status_code": 410
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/private*"
+													]
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"error": "Not found",
+													"handler": "error",
+													"status_code": 404
+												}
+											],
+											"match": [
+												{
+													"path": [
+														"/hidden*"
+													]
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					],
+					"errors": {
+						"routes": [
+							{
+								"match": [
+									{
+										"host": [
+											"localhost"
+										]
+									}
+								],
+								"handle": [
+									{
+										"handler": "subroute",
+										"routes": [
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"handle": [
+																	{
+																		"body": "Error in the [400 .. 499] range",
+																		"handler": "static_response"
+																	}
+																]
+															}
+														]
+													}
+												],
+												"match": [
+													{
+														"expression": "{http.error.status_code} \u003e= 400 \u0026\u0026 {http.error.status_code} \u003c= 499"
+													}
+												]
+											},
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"handle": [
+																	{
+																		"body": "Fallback route: code outside the [400..499] range",
+																		"handler": "static_response"
+																	}
+																]
+															}
+														]
+													}
+												]
+											}
+										]
+									}
+								],
+								"terminal": true
+							}
+						]
+					}
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/error_subhandlers.caddyfiletest

@@ -0,0 +1,260 @@
+{
+	http_port 2099
+}
+localhost:2099 {
+	root * /var/www/
+	file_server
+
+	handle_errors 404 {
+		handle /en/* {
+			respond "not found" 404
+		}
+		handle /es/* {
+			respond "no encontrado"
+		}
+		handle {
+			respond "default not found"
+		}
+	}
+	handle_errors {
+		handle /en/* {
+			respond "English error"
+		}
+		handle /es/* {
+			respond "Spanish error"
+		}
+		handle {
+			respond "Default error"
+		}
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"http_port": 2099,
+			"servers": {
+				"srv0": {
+					"listen": [
+						":2099"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"localhost"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"handler": "vars",
+													"root": "/var/www/"
+												},
+												{
+													"handler": "file_server",
+													"hide": [
+														"./Caddyfile"
+													]
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					],
+					"errors": {
+						"routes": [
+							{
+								"match": [
+									{
+										"host": [
+											"localhost"
+										]
+									}
+								],
+								"handle": [
+									{
+										"handler": "subroute",
+										"routes": [
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"group": "group3",
+																"handle": [
+																	{
+																		"handler": "subroute",
+																		"routes": [
+																			{
+																				"handle": [
+																					{
+																						"body": "not found",
+																						"handler": "static_response",
+																						"status_code": 404
+																					}
+																				]
+																			}
+																		]
+																	}
+																],
+																"match": [
+																	{
+																		"path": [
+																			"/en/*"
+																		]
+																	}
+																]
+															},
+															{
+																"group": "group3",
+																"handle": [
+																	{
+																		"handler": "subroute",
+																		"routes": [
+																			{
+																				"handle": [
+																					{
+																						"body": "no encontrado",
+																						"handler": "static_response"
+																					}
+																				]
+																			}
+																		]
+																	}
+																],
+																"match": [
+																	{
+																		"path": [
+																			"/es/*"
+																		]
+																	}
+																]
+															},
+															{
+																"group": "group3",
+																"handle": [
+																	{
+																		"handler": "subroute",
+																		"routes": [
+																			{
+																				"handle": [
+																					{
+																						"body": "default not found",
+																						"handler": "static_response"
+																					}
+																				]
+																			}
+																		]
+																	}
+																]
+															}
+														]
+													}
+												],
+												"match": [
+													{
+														"expression": "{http.error.status_code} in [404]"
+													}
+												]
+											},
+											{
+												"handle": [
+													{
+														"handler": "subroute",
+														"routes": [
+															{
+																"group": "group8",
+																"handle": [
+																	{
+																		"handler": "subroute",
+																		"routes": [
+																			{
+																				"handle": [
+																					{
+																						"body": "English error",
+																						"handler": "static_response"
+																					}
+																				]
+																			}
+																		]
+																	}
+																],
+																"match": [
+																	{
+																		"path": [
+																			"/en/*"
+																		]
+																	}
+																]
+															},
+															{
+																"group": "group8",
+																"handle": [
+																	{
+																		"handler": "subroute",
+																		"routes": [
+																			{
+																				"handle": [
+																					{
+																						"body": "Spanish error",
+																						"handler": "static_response"
+																					}
+																				]
+																			}
+																		]
+																	}
+																],
+																"match": [
+																	{
+																		"path": [
+																			"/es/*"
+																		]
+																	}
+																]
+															},
+															{
+																"group": "group8",
+																"handle": [
+																	{
+																		"handler": "subroute",
+																		"routes": [
+																			{
+																				"handle": [
+																					{
+																						"body": "Default error",
+																						"handler": "static_response"
+																					}
+																				]
+																			}
+																		]
+																	}
+																]
+															}
+														]
+													}
+												]
+											}
+										]
+									}
+								],
+								"terminal": true
+							}
+						]
+					}
+				}
+			}
+		}
+	}
+}
+

caddytest/integration/caddyfile_adapt/expression_quotes.caddyfiletest

@@ -1,3 +1,7 @@
+(snippet) {
+	@g `{http.error.status_code} == 404`
+}
+
 example.com
 
 @a expression {http.error.status_code} == 400
@@ -14,6 +18,12 @@ abort @d
 
 @e expression `{http.error.status_code} == 404`
 abort @e
+
+@f `{http.error.status_code} == 404`
+abort @f
+
+import snippet
+abort @g
 ----------
 {
 	"apps": {
@@ -84,7 +94,10 @@ abort @e
 											],
 											"match": [
 												{
-													"expression": "{http.error.status_code} == 403"
+													"expression": {
+														"expr": "{http.error.status_code} == 403",
+														"name": "d"
+													}
 												}
 											]
 										},
@@ -97,7 +110,42 @@ abort @e
 											],
 											"match": [
 												{
-													"expression": "{http.error.status_code} == 404"
+													"expression": {
+														"expr": "{http.error.status_code} == 404",
+														"name": "e"
+													}
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"abort": true,
+													"handler": "static_response"
+												}
+											],
+											"match": [
+												{
+													"expression": {
+														"expr": "{http.error.status_code} == 404",
+														"name": "f"
+													}
+												}
+											]
+										},
+										{
+											"handle": [
+												{
+													"abort": true,
+													"handler": "static_response"
+												}
+											],
+											"match": [
+												{
+													"expression": {
+														"expr": "{http.error.status_code} == 404",
+														"name": "g"
+													}
 												}
 											]
 										}

caddytest/integration/caddyfile_adapt/file_server_etag_file_extensions.caddyfiletest

@@ -0,0 +1,40 @@
+:8080 {
+	root * ./
+	file_server {
+		etag_file_extensions .b3sum .sha256
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":8080"
+					],
+					"routes": [
+						{
+							"handle": [
+								{
+									"handler": "vars",
+									"root": "./"
+								},
+								{
+									"etag_file_extensions": [
+										".b3sum",
+										".sha256"
+									],
+									"handler": "file_server",
+									"hide": [
+										"./Caddyfile"
+									]
+								}
+							]
+						}
+					]
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/file_server_file_limit.caddyfiletest

@@ -0,0 +1,36 @@
+:80
+
+file_server {
+	browse {
+		file_limit 4000
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":80"
+					],
+					"routes": [
+						{
+							"handle": [
+								{
+									"browse": {
+										"file_limit": 4000
+									},
+									"handler": "file_server",
+									"hide": [
+										"./Caddyfile"
+									]
+								}
+							]
+						}
+					]
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/file_server_precompressed.caddyfiletest

@@ -3,6 +3,10 @@
 file_server {
 	precompressed zstd br gzip
 }
+
+file_server {
+	precompressed
+}
 ----------
 {
 	"apps": {
@@ -30,6 +34,22 @@ file_server {
 										"br",
 										"gzip"
 									]
+								},
+								{
+									"handler": "file_server",
+									"hide": [
+										"./Caddyfile"
+									],
+									"precompressed": {
+										"br": {},
+										"gzip": {},
+										"zstd": {}
+									},
+									"precompressed_order": [
+										"br",
+										"zstd",
+										"gzip"
+									]
 								}
 							]
 						}

caddytest/integration/caddyfile_adapt/file_server_sort.caddyfiletest

@@ -0,0 +1,39 @@
+:80
+
+file_server {
+	browse {
+		sort size desc
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":80"
+					],
+					"routes": [
+						{
+							"handle": [
+								{
+									"browse": {
+										"sort": [
+											"size",
+											"desc"
+										]
+									},
+									"handler": "file_server",
+									"hide": [
+										"./Caddyfile"
+									]
+								}
+							]
+						}
+					]
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/forward_auth_authelia.caddyfiletest

@@ -0,0 +1,203 @@
+app.example.com {
+	forward_auth authelia:9091 {
+		uri /api/authz/forward-auth
+		copy_headers Remote-User Remote-Groups Remote-Name Remote-Email
+	}
+
+	reverse_proxy backend:8080
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":443"
+					],
+					"routes": [
+						{
+							"match": [
+								{
+									"host": [
+										"app.example.com"
+									]
+								}
+							],
+							"handle": [
+								{
+									"handler": "subroute",
+									"routes": [
+										{
+											"handle": [
+												{
+													"handle_response": [
+														{
+															"match": {
+																"status_code": [
+																	2
+																]
+															},
+															"routes": [
+																{
+																	"handle": [
+																		{
+																			"handler": "vars"
+																		}
+																	]
+																},
+																{
+																	"handle": [
+																		{
+																			"handler": "headers",
+																			"request": {
+																				"set": {
+																					"Remote-Email": [
+																						"{http.reverse_proxy.header.Remote-Email}"
+																					]
+																				}
+																			}
+																		}
+																	],
+																	"match": [
+																		{
+																			"not": [
+																				{
+																					"vars": {
+																						"{http.reverse_proxy.header.Remote-Email}": [
+																							""
+																						]
+																					}
+																				}
+																			]
+																		}
+																	]
+																},
+																{
+																	"handle": [
+																		{
+																			"handler": "headers",
+																			"request": {
+																				"set": {
+																					"Remote-Groups": [
+																						"{http.reverse_proxy.header.Remote-Groups}"
+																					]
+																				}
+																			}
+																		}
+																	],
+																	"match": [
+																		{
+																			"not": [
+																				{
+																					"vars": {
+																						"{http.reverse_proxy.header.Remote-Groups}": [
+																							""
+																						]
+																					}
+																				}
+																			]
+																		}
+																	]
+																},
+																{
+																	"handle": [
+																		{
+																			"handler": "headers",
+																			"request": {
+																				"set": {
+																					"Remote-Name": [
+																						"{http.reverse_proxy.header.Remote-Name}"
+																					]
+																				}
+																			}
+																		}
+																	],
+																	"match": [
+																		{
+																			"not": [
+																				{
+																					"vars": {
+																						"{http.reverse_proxy.header.Remote-Name}": [
+																							""
+																						]
+																					}
+																				}
+																			]
+																		}
+																	]
+																},
+																{
+																	"handle": [
+																		{
+																			"handler": "headers",
+																			"request": {
+																				"set": {
+																					"Remote-User": [
+																						"{http.reverse_proxy.header.Remote-User}"
+																					]
+																				}
+																			}
+																		}
+																	],
+																	"match": [
+																		{
+																			"not": [
+																				{
+																					"vars": {
+																						"{http.reverse_proxy.header.Remote-User}": [
+																							""
+																						]
+																					}
+																				}
+																			]
+																		}
+																	]
+																}
+															]
+														}
+													],
+													"handler": "reverse_proxy",
+													"headers": {
+														"request": {
+															"set": {
+																"X-Forwarded-Method": [
+																	"{http.request.method}"
+																],
+																"X-Forwarded-Uri": [
+																	"{http.request.uri}"
+																]
+															}
+														}
+													},
+													"rewrite": {
+														"method": "GET",
+														"uri": "/api/authz/forward-auth"
+													},
+													"upstreams": [
+														{
+															"dial": "authelia:9091"
+														}
+													]
+												},
+												{
+													"handler": "reverse_proxy",
+													"upstreams": [
+														{
+															"dial": "backend:8080"
+														}
+													]
+												}
+											]
+										}
+									]
+								}
+							],
+							"terminal": true
+						}
+					]
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/forward_auth_authelia.txt

@@ -1,111 +0,0 @@
-app.example.com {
-	forward_auth authelia:9091 {
-		uri /api/verify?rd=https://authelia.example.com
-		copy_headers Remote-User Remote-Groups Remote-Name Remote-Email
-	}
-
-	reverse_proxy backend:8080
-}
-----------
-{
-	"apps": {
-		"http": {
-			"servers": {
-				"srv0": {
-					"listen": [
-						":443"
-					],
-					"routes": [
-						{
-							"match": [
-								{
-									"host": [
-										"app.example.com"
-									]
-								}
-							],
-							"handle": [
-								{
-									"handler": "subroute",
-									"routes": [
-										{
-											"handle": [
-												{
-													"handle_response": [
-														{
-															"match": {
-																"status_code": [
-																	2
-																]
-															},
-															"routes": [
-																{
-																	"handle": [
-																		{
-																			"handler": "headers",
-																			"request": {
-																				"set": {
-																					"Remote-Email": [
-																						"{http.reverse_proxy.header.Remote-Email}"
-																					],
-																					"Remote-Groups": [
-																						"{http.reverse_proxy.header.Remote-Groups}"
-																					],
-																					"Remote-Name": [
-																						"{http.reverse_proxy.header.Remote-Name}"
-																					],
-																					"Remote-User": [
-																						"{http.reverse_proxy.header.Remote-User}"
-																					]
-																				}
-																			}
-																		}
-																	]
-																}
-															]
-														}
-													],
-													"handler": "reverse_proxy",
-													"headers": {
-														"request": {
-															"set": {
-																"X-Forwarded-Method": [
-																	"{http.request.method}"
-																],
-																"X-Forwarded-Uri": [
-																	"{http.request.uri}"
-																]
-															}
-														}
-													},
-													"rewrite": {
-														"method": "GET",
-														"uri": "/api/verify?rd=https://authelia.example.com"
-													},
-													"upstreams": [
-														{
-															"dial": "authelia:9091"
-														}
-													]
-												},
-												{
-													"handler": "reverse_proxy",
-													"upstreams": [
-														{
-															"dial": "backend:8080"
-														}
-													]
-												}
-											]
-										}
-									]
-								}
-							],
-							"terminal": true
-						}
-					]
-				}
-			}
-		}
-	}
-}

caddytest/integration/caddyfile_adapt/forward_auth_rename_headers.caddyfiletest

@@ -0,0 +1,206 @@
+:8881
+
+forward_auth localhost:9000 {
+	uri /auth
+	copy_headers A>1 B C>3 {
+		D
+		E>5
+	}
+}
+----------
+{
+	"apps": {
+		"http": {
+			"servers": {
+				"srv0": {
+					"listen": [
+						":8881"
+					],
+					"routes": [
+						{
+							"handle": [
+								{
+									"handle_response": [
+										{
+											"match": {
+												"status_code": [
+													2
+												]
+											},
+											"routes": [
+												{
+													"handle": [
+														{
+															"handler": "vars"
+														}
+													]
+												},
+												{
+													"handle": [
+														{
+															"handler": "headers",
+															"request": {
+																"set": {
+																	"1": [
+																		"{http.reverse_proxy.header.A}"
+																	]
+																}
+															}
+														}
+													],
+													"match": [
+														{
+															"not": [
+																{
+																	"vars": {
+																		"{http.reverse_proxy.header.A}": [
+																			""
+																		]
+																	}
+																}
+															]
+														}
+													]
+												},
+												{
+													"handle": [
+														{
+															"handler": "headers",
+															"request": {
+																"set": {
+																	"B": [
+																		"{http.reverse_proxy.header.B}"
+																	]
+																}
+															}
+														}
+													],
+													"match": [
+														{
+															"not": [
+																{
+																	"vars": {
+																		"{http.reverse_proxy.header.B}": [
+																			""
+																		]
+																	}
+																}
+															]
+														}
+													]
+												},
+												{
+													"handle": [
+														{
+															"handler": "headers",
+															"request": {
+																"set": {
+																	"3": [
+																		"{http.reverse_proxy.header.C}"
+																	]
+																}
+															}
+														}
+													],
+													"match": [
+														{
+															"not": [
+																{
+																	"vars": {
+																		"{http.reverse_proxy.header.C}": [
+																			""
+																		]
+																	}
+																}
+															]
+														}
+													]
+												},
+												{
+													"handle": [
+														{
+															"handler": "headers",
+															"request": {
+																"set": {
+																	"D": [
+																		"{http.reverse_proxy.header.D}"
+																	]
+																}
+															}
+														}
+													],
+													"match": [
+														{
+															"not": [
+																{
+																	"vars": {
+																		"{http.reverse_proxy.header.D}": [
+																			""
+																		]
+																	}
+																}
+															]
+														}
+													]
+												},
+												{
+													"handle": [
+														{
+															"handler": "headers",
+															"request": {
+																"set": {
+																	"5": [
+																		"{http.reverse_proxy.header.E}"
+																	]
+																}
+															}
+														}
+													],
+													"match": [
+														{
+															"not": [
+																{
+																	"vars": {
+																		"{http.reverse_proxy.header.E}": [
+																			""
+																		]
+																	}
+																}
+															]
+														}
+													]
+												}
+											]
+										}
+									],
+									"handler": "reverse_proxy",
+									"headers": {
+										"request": {
+											"set": {
+												"X-Forwarded-Method": [
+													"{http.request.method}"
+												],
+												"X-Forwarded-Uri": [
+													"{http.request.uri}"
+												]
+											}
+										}
+									},
+									"rewrite": {
+										"method": "GET",
+										"uri": "/auth"
+									},
+									"upstreams": [
+										{
+											"dial": "localhost:9000"
+										}
+									]
+								}
+							]
+						}
+					]
+				}
+			}
+		}
+	}
+}

caddytest/integration/caddyfile_adapt/forward_auth_rename_headers.txt

@@ -1,90 +0,0 @@
-:8881
-
-forward_auth localhost:9000 {
-	uri /auth
-	copy_headers A>1 B C>3 {
-		D
-		E>5
-	}
-}
-----------
-{
-	"apps": {
-		"http": {
-			"servers": {
-				"srv0": {
-					"listen": [
-						":8881"
-					],
-					"routes": [
-						{
-							"handle": [
-								{
-									"handle_response": [
-										{
-											"match": {
-												"status_code": [
-													2
-												]
-											},
-											"routes": [
-												{
-													"handle": [
-														{
-															"handler": "headers",
-															"request": {
-																"set": {
-																	"1": [
-																		"{http.reverse_proxy.header.A}"
-																	],
-																	"3": [
-																		"{http.reverse_proxy.header.C}"
-																	],
-																	"5": [
-																		"{http.reverse_proxy.header.E}"
-																	],
-																	"B": [
-																		"{http.reverse_proxy.header.B}"
-																	],
-																	"D": [
-																		"{http.reverse_proxy.header.D}"
-																	]
-																}
-															}
-														}
-													]
-												}
-											]
-										}
-									],
-									"handler": "reverse_proxy",
-									"headers": {
-										"request": {
-											"set": {
-												"X-Forwarded-Method": [
-													"{http.request.method}"
-												],
-												"X-Forwarded-Uri": [
-													"{http.request.uri}"
-												]
-											}
-										}
-									},
-									"rewrite": {
-										"method": "GET",
-										"uri": "/auth"
-									},
-									"upstreams": [
-										{
-											"dial": "localhost:9000"
-										}
-									]
-								}
-							]
-						}
-					]
-				}
-			}
-		}
-	}
-}

caddytest/integration/caddyfile_adapt/global_options.caddyfiletest

@@ -9,6 +9,8 @@
 	storage file_system {
 		root /data
 	}
+	storage_check off
+	storage_clean_interval off
 	acme_ca https://example.com
 	acme_ca_root /path/to/ca.crt
 	ocsp_stapling off
@@ -17,8 +19,6 @@
 	admin off
 	on_demand_tls {
 		ask https://example.com
-		interval 30s
-		burst 20
 	}
 	local_certs
 	key_type ed25519
@@ -69,14 +69,15 @@
 					}
 				],
 				"on_demand": {
-					"rate_limit": {
-						"interval": 30000000000,
-						"burst": 20
-					},
-					"ask": "https://example.com"
+					"permission": {
+						"endpoint": "https://example.com",
+						"module": "http"
+					}
 				}
 			},
-			"disable_ocsp_stapling": true
+			"disable_ocsp_stapling": true,
+			"disable_storage_check": true,
+			"disable_storage_clean": true
 		}
 	}
 }

caddytest/integration/caddyfile_adapt/global_options_acme.caddyfiletest

@@ -17,8 +17,6 @@
 	admin off
 	on_demand_tls {
 		ask https://example.com
-		interval 30s
-		burst 20
 	}
 	storage_clean_interval 7d
 	renew_interval 1d
@@ -63,6 +61,14 @@
 						"issuers": [
 							{
 								"ca": "https://example.com",
+								"challenges": {
+									"http": {
+										"alternate_port": 8080
+									},
+									"tls-alpn": {
+										"alternate_port": 8443
+									}
+								},
 								"email": "test@example.com",
 								"external_account": {
 									"key_id": "4K2scIVbBpNd-78scadB2g",
@@ -78,11 +84,10 @@
 					}
 				],
 				"on_demand": {
-					"rate_limit": {
-						"interval": 30000000000,
-						"burst": 20
-					},
-					"ask": "https://example.com"
+					"permission": {
+						"endpoint": "https://example.com",
+						"module": "http"
+					}
 				},
 				"ocsp_interval": 172800000000000,
 				"renew_interval": 86400000000000,

caddytest/integration/caddyfile_adapt/global_options_admin.caddyfiletest

@@ -16,8 +16,6 @@
 	}
 	on_demand_tls {
 		ask https://example.com
-		interval 30s
-		burst 20
 	}
 	local_certs
 	key_type ed25519
@@ -71,11 +69,10 @@
 					}
 				],
 				"on_demand": {
-					"rate_limit": {
-						"interval": 30000000000,
-						"burst": 20
-					},
-					"ask": "https://example.com"
+					"permission": {
+						"endpoint": "https://example.com",
+						"module": "http"
+					}
 				}
 			}
 		}

caddytest/integration/caddyfile_adapt/global_options_admin_with_persist_config_off.caddyfiletest

@@ -0,0 +1,36 @@
+{
+	http_port 8080
+	persist_config off
+	admin {
+		origins localhost:2019 [::1]:2019 127.0.0.1:2019 192.168.10.128
+	}
+}
+
+:80
+----------
+{
+	"admin": {
+		"listen": "localhost:2019",
+		"origins": [
+			"localhost:2019",
+			"[::1]:2019",
+			"127.0.0.1:2019",
+			"192.168.10.128"
+		],
+		"config": {
+			"persist": false
+		}
+	},
+	"apps": {
+		"http": {
+			"http_port": 8080,
+			"servers": {
+				"srv0": {
+					"listen": [
+						":80"
+					]
+				}
+			}
+		}
+	}
+}