Elasticsearch is a distributed search engine designed for speed, scalability, and reliability. Built on Apache Lucene, it allows you to store, search, and analyze large volumes of data in near real-time.

Elasticsearch is widely used for applications like website and ecommerce search, making it a popular choice for enterprises needing robust search capabilities.

Elasticsearch is one of our most successful managed services, powering the search of hundreds of projects. Our products broadly support Elasticsearch, from version 1.4 to the latest 8.x releases.

Feel free to join us on our Discord, or any other ways to discuss this with our team.

The issue at hand

Over the years, the Elasticsearch licenses have evolved:

Before 2021: the majority of the code was under an Apache 2.0 license.
After 2021, dual-licensed: The Apache 2.0 licensed code changed to a dual license under the Service Side Public License (SSPL) and the Elastic License (EL).\

In August 2024, triple-licensed: The GNU Affero General Public License (AGPLV3) is another option, alongside ELv2 and SSPL.

All our services are offered as managed services. From that perspective, here are the main differences between the four licenses:

• Apache 2.0: A permissive license that can be provided as a managed service.
• SSPL: A server side public license that requires releasing the source code for the entirety of the service. This includes all “management software, user interfaces, application program interfaces, automation software, monitoring software, backup software, storage software and hosting software, all such that a user could run an instance of the service using the Service Source Code you make available”. For Upsun, this means all our tooling must be released to manage the service. The SSPL is the license used by MongoDB and now Redis too.
• AGPLV3: This license is compatible with managed services and requires that any modifications made to softwares under the same license must also be published.
• Elastic License V2: a commercial license that is compatible with reselling Elasticsearch as a managed service.

That gives us the following landscape:

• All versions below 7.10: all of these versions have been deprecated for several years but are still available to be used with no additional charge.
• Elasticsearch 7.17 and 8.5: There is an additional charge when using Elasticsearch 7.17 and 8.5 as these are classed as premium services.

After the latest license change, we thought it was going to be a game changer for our customers, with several new AI features to be leveraged.

The licensing issue to using new Elasticsearch releases

Our customers asked us to provide Elasticsearch 8.16, free of charge, so we started to do our homework to provide it.

At Upsun, services are built using internal tooling, driven by a configuration file with multiple attributes and the required packages. For elasticsearch “premium”, it looks like this:

elasticsearch:
	url: https://artifacts.elastic.co/packages/8.x/apt

packages:
	elasticsearch: "= 8.5.3"

So for elasticsearch 8.16, we thought it would be as easy as replacing 8.5.3 by 8.16.x. That would be it. But you need to pay attention to one sentence on the “Elastic FAQ on Software Licensing” page.

The sentence that caught our attention states the following: “Our releases will continue to be under the Elastic License.”. It should be noted that the Elastic FAQ page is the go-to page when you have questions about Elasticsearch Licensing. It has existed since the SSPL switch. We verified the download page and confirmed the distributed software uses Elastic License 2.0.

It was also confirmed by a post in the Elastic forum that they: “do not currently offer any downloadable distribution that is covered by the AGPL.”

At that point we knew that if we wanted to support Elasticsearch again, we would need to put in more effort as we can’t use the prebuilt releases.

Let’s build it!

In the images team, we are a bit stubborn. If we can’t use the available packages, we build our own. It’s not because it’s in Java, using Gradle that we can’t do it.

We started by a nice discussion with our legal team, to discuss what we could do and agreed on:

• build Elasticsearch from the unmodified sources
• publish the resulting Debian package under AGPL V3 in a publicly accessible Github repository

Our engineering team cloned the repository and started triggering the Gradle tooling.

Surprisingly, they didn’t create an AGPL V3 build. It appears that the code to build the OSS flavor of Elasticsearch has been removed, as confirmed by the following pull requests:

• Remove all OSS distribution references from Elasticsearch build #68797
• Remove OSS specific distribution packaging #69775

We dug a bit, and the original work was sent upstream in pull requests by AWS as explained here and removed in the previous license change:

“changes to Elasticsearch were sent as upstream pull requests (#42066, #42658, #43284, #43839, #53643, #57271, #59563, #61400, #64513), and we then included the “oss” builds offered by Elastic in our distribution. This ensured that we were collaborating with the upstream developers and maintainers, and not creating a “fork” of the software.”

So, it’s not easy to build an AGPL v3 package for Elasticsearch; the code simply doesn’t have the tooling.

Let’s modify the build tooling!

We went back to our legal team, to discuss what we could do, and agreed on:

• fork the Elasticsearch repository
• update the Gradle tooling to build an OSS distribution
• on a public repository, under AGPL V3:
• share all the code
• share the build artifacts
• send an email to Elasticsearch explaining what we did

The source code is not consistent in terms of license; some parts are under a triple license, including the open source license AGPL v3, and some are under the non-open source Elastic License 2.0. We are not allowed to provide a service with non-open source licenses, thus our only choice was to remove the following two folders from the build:

• x-pack
• apm

Have you met x-pack?

x-pack is an Elastic extension that provides various capabilities, including the security layer. It has been in Elasticsearch for a long time, as a paid extension, and it has a “turbulent” life of its own: “Doubling Down on Open”.

As previously mentioned, we had to remove the x-pack folder to build an open source package.

However, once we removed x-pack, Elasticsearch wouldn’t start anymore; the code is coupled to x-pack’s security features. You can’t turn that off.

root@644e331111:/mnt# /usr/share/elasticsearch/bin/elasticsearch
java.io.UncheckedIOException: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/modules/x-pack-core
    	at org.elasticsearch.cli.CliToolProvider.loadJars(CliToolProvider.java:87)
    	at org.elasticsearch.cli.CliToolProvider.load(CliToolProvider.java:59)
    	at org.elasticsearch.server.cli.ServerCli.loadTool(ServerCli.java:262)
    	at org.elasticsearch.server.cli.ServerCli.autoConfigureSecurity(ServerCli.java:171)
    	at org.elasticsearch.server.cli.ServerCli.execute(ServerCli.java:91)
    	at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:55)
    	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:95)
    	at org.elasticsearch.cli.Command.main(Command.java:52)
    	at org.elasticsearch.launcher.CliToolLauncher.main(CliToolLauncher.java:65)
Caused by: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/modules/x-pack-core
    	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
    	at java.base/sun.nio.fs.UnixException.asIOException(UnixException.java:115)
    	at java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:502)
    	at java.base/java.nio.file.Files.newDirectoryStream(Files.java:482)
    	at java.base/java.nio.file.Files.list(Files.java:3796)
    	at org.elasticsearch.cli.CliToolProvider.loadJars(CliToolProvider.java:78)
    	... 8 more

But as we investigated more, we found a way to tweak the code make it work without x-pack.

Secure, very secure

We noticed Elasticsearch sets up security before executing the server. The code below calls secureSettingsLoader.supportsSecurityAutoConfiguration() to decide if it enables security or not.

try (
   var loadedSecrets = secureSettingsLoader.load(env, terminal);
   var password = (loadedSecrets.password().isPresent()) ? loadedSecrets.password().get() : new SecureString(new char[0]);
) {
   SecureSettings secrets = loadedSecrets.secrets();
   if (secureSettingsLoader.supportsSecurityAutoConfiguration()) {
   	env = autoConfigureSecurity(terminal, options, processInfo, env, password);
   	// reload or create the secrets
   	secrets = secureSettingsLoader.bootstrap(env, password);
   }

However, the implementation of the method could not be anything but a surprise. The method always returns true! Elasticsearch is very secure, indeed.

public boolean supportsSecurityAutoConfiguration() {
   return true;
}

We were not ready to give up yet. It seemed just a simple matter of overriding it to return false. Well, after preventing the security configuration, all hell broke loose again. The Elasticsearch server expects secrets to be in place. It does not even let you pass secrets as null, as we can see from the code snippet below.

public ServerArgs {
   assert pidFile == null || pidFile.isAbsolute();
   assert secrets != null;
}

The assert does not allow secrets to be null. Unfortunately, when security is not configured, the secrets ends up being null!

Finding ourselves at a crossroads again, we realized something.

Wait, are we really forking Elasticsearch?

At that point, we took a step back, looking at what we were doing. We realized that “we are forking Elasticsearch”:

• We forked the repository
• We deeply modified the code, involving complicated maintenance and version upgrades
• We were proposing a new build

It probably can’t be named Elasticsearch anymore; it’s more than just different tooling, it’s a different software altogether.

Let’s not fork it?

We stopped our efforts and decided to publish this blog post, communicating openly about the current status.

It’s not our business to maintain a fork of Elasticsearch; it will require a new name and the maintenance cost will be huge: for each version, an engineer needs to update the code.

And honestly, we do not want to fork it.

It’s not the end of the journey; the team is ready to reopen the project at any time. The distribution license may change, or the OSS build added back to the repository.

And remember: Upsun ❤️ Elasticsearch