Choosing HTTP server for evitaDB

Criteria and requirements

As the main requirement, we wanted the lowest possible latency and the highest possible throughput of HTTP request processing and also a simple codebase of chosen server, library or framework so that there would be a smaller probability of “magic” behavior and unexpected surprises. Another important feature was the ability to embed the HTTP foundation into the existing evitaDB codebase without needing to adjust the whole running and building processes of evitaDB to the chosen solution because in future there, may be other ways to communicate with users. An advantage to all of these requirements would be a simple and straightforward API for handling HTTP requests and errors. This means that there wouldn't be any unnecessary low level HTTP communication work involved, if not explicitly needed by some specific use-cases. Last but not least, it would be nice to have at least partial built-in support for handling WebSockets for future specific use-cases. Finally, a server or library that is publicly known to work or being tested with GraalVM or having direct GraalVM support is a plus.

Servers, libraries and frameworks

We decided to spend up to 8 hours on ecosystem exploration and chose the following 10 Java HTTP servers and frameworks to test:

• microhttp
• NanoHTTPD
• Netty
• Undertow
• Spring MVC with Spring Boot (runs on Tomcat, Jetty or Undertow, we used Tomcat as it is the default)
• Spring WebFlux with Spring Boot (runs on Tomcat, Jetty or Undertow, we used Tomcat as it is the default)
• Vert.x (runs on Netty)
• Quarkus Native (runs on Netty through Vert.x)
• Micronaut (runs on Netty)
• Javalin (runs on Jetty)

This list contains low level servers, small libraries and even big and well known frameworks for building web applications. We decided to include those big frameworks to have some performance comparison with lower level servers so that we know whether the possible sacrifice of missing high level abstractions is really worth it. Our servers, libraries and frameworks were selected upon several recommendation articles and mainly by GitHub repository popularity, i.e. star count, issues, dates of last commits and so on.

Testing environment

We tested the servers, libraries and frameworks on a simple “echo” GraphQL API because we wanted to test the baseline latency of HTTP request processing for GraphQL API. The GraphQL API was chosen because it is the first API evitaDB is going to support, and we believe that this approach will give us the accurate measurement even for other future APIs. This simplified test API was then implemented on top of each chosen server, library and framework.

The “echo” API contains single query that takes single string argument:

which it then returns in response message:

No additional business logic is implemented there.

Implementation of servers

Benchmarking servers

The final tests were run with 1 warm-up iteration, 5 measurement iterations and 2 forks on a laptop with Ubuntu 21.10 and an 8-core Intel Core i7-8550U CPU with 16 GB of RAM. The tests were run in two modes: throughput (operations per second) and average time (microseconds per operation).

Choosing the final solution

Conclusion

Every single server, library or framework was successfully used to build a server with the example “echo” API. That means that, functionally, all of these solutions would be sufficient. But when we incorporated requirements for performance and embedding possibilities, possible servers, libraries and frameworks narrowed. For example, Spring Boot solutions required its own build Maven plugins and workflow, the Quarkus solution also required its own workflow and we couldn’t even make it work across multiple Maven modules, although it should be possible. Micronaut was probably the worst when it came to setting it up, because not only did it require its own code structure and custom Maven plugin, but also required its own Maven parent POM which is quite a problem in multi-module projects like evitaDB. This shortcoming could probably be overcome by investing more time to deeply understand the required dependencies, but we fear that if simple setup is that difficult, there may be other surprises that could arise in the future. Other solutions were embedded fairly easily. One exception was the Netty server. Implementation of a server on Netty required a lot of low level HTTP configuring (which for new users seems almost like gibberish). Another difficulty was the enormous amount of different HTTP request and response classes that are somehow combined to the final ones. But this is probably due to the universality of the whole Netty ecosystem.