Embedded Tomcat Internals and Tuning

Spring Boot starts Tomcat inside your application. It is important to know how a request travels through Tomcat and how threads and connections are limited, because most production performance problems and 'connection refused' or 'timeout' errors come from these limits.

Tomcat architecture

Server
 └─ Service
     ├─ Connector (NIO, port 8080)  ── handles the network
     │     ├─ Acceptor thread   : accepts new TCP connections
     │     ├─ Poller thread(s)  : watches sockets, finds those with data (NIO selector)
     │     └─ Worker threads    : "http-nio-8080-exec-N", run the servlet code   (pool size = server.tomcat.threads.max)
     └─ Engine ─► Host ─► Context (your application)
                                 └─ Filters ─► DispatcherServlet ─► your @RestController

Life of a request

1. Client opens a TCP connection            ─► accept queue (accept-count, default 100, when max-connections is reached)
2. Acceptor accepts it                       ─► counted in max-connections (default 8192)
3. Poller waits until request bytes arrive   (idle keep-alive connections cost no thread)
4. A worker thread is taken from the pool    ─► request parsed, filters, controller, response written
5. Thread returns to the pool; the connection stays open for keep-alive
If all 200 worker threads are busy, new requests WAIT in the queue. Slow requests = fewer free threads = higher latency.

Important properties

How many threads do you need? By Little's law: concurrent requests = requests per second x average duration in seconds. At 500 requests/second and 200 ms average, about 100 threads are busy. If a downstream call becomes slow (2 seconds), 500 x 2 = 1000 threads would be needed, so the pool is exhausted and the whole service looks down. This is why timeouts and circuit breakers matter.

Common configuration

server.port=8080
server.tomcat.threads.max=300
server.tomcat.accept-count=200
server.compression.enabled=true
server.compression.mime-types=application/json,text/html,text/css,application/javascript
server.compression.min-response-size=1024
server.shutdown=graceful
server.forward-headers-strategy=framework      # behind a load balancer or proxy: trust X-Forwarded-* headers
server.tomcat.accesslog.enabled=true
server.tomcat.accesslog.directory=logs
server.error.include-message=never
server.error.include-stacktrace=never

HTTPS (SSL bundles, Boot 3.1+)

spring.ssl.bundle.jks.web.keystore.location=classpath:keystore.p12
spring.ssl.bundle.jks.web.keystore.password=${KEYSTORE_PASSWORD}
spring.ssl.bundle.jks.web.keystore.type=PKCS12
server.ssl.bundle=web
server.port=8443
server.http2.enabled=true                       # HTTP/2 needs TLS in browsers

In production TLS is usually terminated by a load balancer, ingress or reverse proxy (NGINX, Envoy, cloud LB), and the Spring Boot application speaks plain HTTP inside the private network.

Customize Tomcat in code

@Component
public class TomcatCustomizer implements WebServerFactoryCustomizer<TomcatServletWebServerFactory> {
    @Override
    public void customize(TomcatServletWebServerFactory factory) {
        factory.addConnectorCustomizers(connector -> {
            connector.setProperty("maxKeepAliveRequests", "200");
            connector.setProperty("keepAliveTimeout", "15000");
        });
        factory.setPort(8080);
    }
}

Virtual threads

With Java 21 and spring.threads.virtual.enabled=true, Tomcat runs each request on a virtual thread. Blocking calls (JDBC, HTTP) no longer hold a costly platform thread, so the 200-thread limit stops being the bottleneck. The new limit becomes your downstream resources: the database connection pool, external APIs and CPU. Size the Hikari pool and use timeouts accordingly.

spring.threads.virtual.enabled=true
# still important: limit concurrent database work
spring.datasource.hikari.maximum-pool-size=20

Read the real port and inspect threads

# when server.port=0 a random free port is used (great for tests)
curl localhost:8080/actuator/metrics/tomcat.threads.busy
curl localhost:8080/actuator/metrics/tomcat.threads.current
curl localhost:8080/actuator/threaddump | head -50
# enable Tomcat MBean metrics:  server.tomcat.mbeanregistry.enabled=true

Common mistake: Raising server.tomcat.threads.max to 2000 to fix slowness. More threads only move the bottleneck to the database or CPU and can make things worse. Find the slow downstream call, add timeouts and limits, then size threads and pools together.

Interview questions

What does the thread http-nio-8080-exec-3 mean?

It is worker thread number 3 of the Tomcat NIO connector on port 8080. It runs your controller code for one request at a time.

Why do I get 400 Request Header Is Too Large?

Headers (often huge cookies or JWTs) exceeded server.max-http-request-header-size (8KB). Reduce the header size or raise the limit.

Tomcat, Jetty or Undertow?

All are fine. Tomcat is the default and the most used. Choose another only for a specific need, and measure before changing.