How to manage the `.well-known` directory on Upsun and Platform.sh

Introduction

If you have ever tried to enable Apple Pay on a website, or needed to verify you control a domain to generate a Let’s Encrypt certificate, or noticed numerous requests to well-known/traffic-advice in your access log, then you’ve probably found yourself working with the .well-known directory at the root of your site. As defined in RFC 8615, .well-known is a special, standardized location at the root of a website that serves as a place where applications, services, or security protocols can find important metadata or configuration files.

In the examples earlier, Apple Pay requests the file apple-developer-merchantid-domain-association inside the .well-known directory in order to verify your merchant domain.
Let’s Encrypt uses .domain-verification inside /.well-known/acme-challenge in order to verify domain control before generating a certificate. The Chrome browser retrieves traffic-advice from inside .well-known in order to determine if the site allows pre-fetching, and if so, how much.

But this is just the tip of the iceberg. There are numerous items that can be placed inside the .well-known directory. And as the number of types of applications, services, and protocols continues to grow, so will the number of files that will be requested from this “well known location”.

The challenge

You may be thinking:

“Can’t I just create a .well-known directory in my repository, add my files, commit, push to Platform.sh/Upsun and be done?”

Yes… and no. The challenge is some of the files that you might need to place in .well-know are extension-less (e.g. apple-app-site-association and traffic-advice), even though they contain specific types of data (json in the two previous examples). If you place an extension-less in your project, the web server is unable to determine the mime type and defaults to content-type: application/octet-stream

❯ curl -I https://example.com/.well-known/apple-app-site-association
HTTP/2 200
accept-ranges: bytes
cache-control: no-cache
content-type: application/octet-stream

So then how can we configure our project to contain extension-less files but also serve the correct mime-type? Luckily, we have a couple of options to utilize to solve this challenge.

Option 1 - Rewrite the Request

The first method is to include the proper extension on the file so the web server knows how to set the correct mime type , and then add a rewrite rule in our locations for requests to the extension-less version that “pass(es)thru” to our version with the extension.

First, add the file to the .well-known directory with the extension in your project:

❯ tree web/.well-known/
web/.well-known/
└── apple-app-site-association.json

Next we’ll adjust the root (/) location in our .platform.app.yaml (Platform.sh) or .upsun/config.yaml (Upsun) configuration file to include the rule:

  locations:
    "/":
      passthru: "/index.php"
      root: "web"
      rules:
        '^/.well-known/apple-app-site-association$':
           allow: true
           passthru: '/.well-known/apple-app-site-association.json'

Now when a request is made for https://example.com/.well-know/apple-app-site-association the web server will rewrite the request to web/.well-known/apple-app-site-association.json, and since it know the file is json, will return the correct mime type:

❯ curl -I https://example.com/.well-known/apple-app-site-association
HTTP/2 200
accept-ranges: bytes
cache-control: no-cache
content-type: application/json

Option 2 - Manually set the Header Response

The second option is when you need even more flexibility in controlling the mime type that is returned. With the example of traffic-advice earlier, not only is the request for an extension-less file, Chrome expects the mime-type to be a specific mime type of application/trafficadvice+json. If not, Chrome will reject it. It is entirely possible that other services also require a specific mime type in the server response, or others may be added in the future with similar requirements. To tackle this challenge we’ll take a similar approach of using rules, but take advantage of some additional properties that are available in the rules section.

Just like Option 1, go ahead and add your file. Given we’re going to manually set the header response for the mime type, you can leave the file extension-less.

❯ tree web/.well-known/
web/.well-known/
├── apple-app-site-association.json
└── traffic-advice

Without adding the rules, a request to the traffic-advice will return the wrong mime type causing Chrome to reject the response:

❯ curl -I https://example.com/.well-known/traffic-advice
HTTP/2 200
accept-ranges: bytes
cache-control: no-cache
content-type: application/octet-stream

Let’s then adjust our location in our .platform.app.yaml (Platform.sh) or .upsun/config.yaml (Upsun) configuration file to respond with a custom header of application/trafficadvice+json when the traffic-advice file is requested:

  locations:
    "/":
      passthru: "/index.php"
      root: "web"
    "/.well-known":
      root: "web/.well-known"
      rules:
        'apple-app-site-association':
          allow: true
          passthru: '/.well-known/apple-app-site-association.json'
        'traffic-advice':
          headers:
            Content-Type: application/trafficadvice+json

After pushing our change, now when something requests our traffic-advice file, the server not only responds with the contents of the file, but is able to send the correct mime type:

❯ curl -I https://example.com/.well-known/traffic-advice
HTTP/2 200
accept-ranges: bytes
cache-control: no-cache
content-type: application/trafficadvice+json

Option 3 - Use a Mount

If desired, or you don’t want to include these types of files in your repository, you could create a mount for .well-known, upload the needed files, and then use any combination of the rules with passthru or custom headers to serve those files exactly as needed.

Conclusion

Platform.sh and Upsun provide exceptional flexibility when configuring your projects, enabling developers to not only fine-tune how extension-less files are served while maintaining correct MIME types, but also any other unexpected item in your day-to-day. Whether leveraging rewrite rules to map requests to explicitly defined files, customizing response headers for specialized formats, or accommodating future services with unique demands, by offering multiple configuration strategies, Platform.sh and Upsun make it easy to scale and maintain high-performance applications without sacrificing control over crucial web protocols.