Measuring API Capacity and Performance in Node.js

1. Introduction

When building a backend application, it is important to know how many requests our API can handle, how long requests take to complete, and when the system begins to slow down.

Without performance metrics, we might increase server resources or database connection pool sizes without understanding the actual problem.

This guide explains how to measure API performance using Node.js, Express, Little's Law, and basic production monitoring techniques.

2. What Is Active Request Count?

Active request count is the number of HTTP requests that have started but have not yet finished their response lifecycle.

Imagine this sequence:

Time	Event	Active requests
12:00:01	Request A arrives	1
12:00:02	Request B arrives	2
12:00:03	Request A finishes	1
12:00:04	Request C arrives	2
12:00:05	Requests B and C finish	0

The server received three requests, but its active request count never exceeded two.

Active requests help us understand how much work is currently in progress.

However, active HTTP requests are not the same as active database queries. A request might be waiting for a database connection, calling another API, or performing other work.

3. Understanding Little's Law

Little's Law describes the relationship between concurrency, throughput, and latency.

Formula:

L = λ × W

Where:

• L = average number of requests in the system.

• λ = average throughput, measured in completed requests per second.

• W = average time spent in the system, measured in seconds.

We can rearrange the formula:

λ = L / W

Example

Suppose an API has:

• Average concurrency: 20 requests.

• Average request duration: 0.2 seconds.

Then:

Throughput = 20 / 0.2 = 100 requests/second

This describes the throughput associated with those average measurements. It does not guarantee that the server can sustain 100 requests per second under every workload.

Why latency matters

Suppose throughput remains at 100 requests per second, but average latency increases from 0.2 seconds to 2 seconds.

Using Little's Law:

• Before: L = 100 × 0.2 = 20

• After: L = 100 × 2 = 200

The system now has approximately 200 requests in progress on average.

This may increase memory usage and resource contention. If a dependency has limited capacity, requests may accumulate in its queue.

4. Instrumenting an Express API

We can measure HTTP request duration and active request count using middleware.

import express from "express";
import { performance } from "node:perf_hooks";

const app = express();

let activeRequests = 0;
let completedRequests = 0;
let totalDurationMs = 0;

app.use((req, res, next) => {
  const start = performance.now();
  let cleanedUp = false;

  activeRequests++;

  const cleanup = () => {
    if (cleanedUp) return;
    cleanedUp = true;

    activeRequests--;

    // Only completed responses count as completed requests.
    if (res.writableFinished) {
      completedRequests++;
      totalDurationMs += performance.now() - start;
    }
  };

  res.once("finish", cleanup);
  res.once("close", cleanup);

  next();
});

app.get("/slow-data", async (req, res) => {
  await new Promise((resolve) => setTimeout(resolve, 2000));
  res.json({ message: "Done" });
});

app.get("/metrics-summary", (req, res) => {
  res.json({
    activeRequests,
    completedRequests,
    averageDurationMs:
      completedRequests > 0
        ? totalDurationMs / completedRequests
        : 0,
  });
});

app.listen(3000, () => {
  console.log("Server listening on port 3000");
});

How this code works

1. When a request arrives, increment activeRequests.

2. Start a high-resolution timer using performance.now().

3. When the response finishes or closes, run cleanup once.

4. Decrement the active request count.

5. Record duration only if the response finished successfully at the HTTP lifecycle level.

Note that finish means the response has been handed off to the underlying system for transmission; it does not guarantee the client received every byte.

This example is suitable for learning. For production use, add request-duration histograms, status-code counts, error metrics, and a proper metrics exporter. Also avoid exposing internal metrics publicly without access controls.

5. How Do We Collect Real Performance Metrics?

We do not need to guess how long a database query takes. We measure it.

HTTP metrics

Collect:

• Request rate: requests per second.

• Request duration: how long requests take.

• Error rate: percentage of requests returning errors.

• In-flight requests: requests currently in progress.

Database metrics

Collect:

• Connection pool size and active connections.

• Idle connections.

• Pending requests waiting for a connection.

• Query duration.

• Slow query counts.

• Database CPU, memory, and lock waits.

Infrastructure metrics

Collect:

• CPU utilization.

• Memory usage and heap growth.

• Event-loop delay.

• Network utilization.

• Container restarts and replica count.

Use monitoring tools such as Prometheus and Grafana, or an application performance monitoring platform, to collect and visualize these measurements.

6. Understanding Database Connection Pool Bottlenecks

Suppose an application has a PostgreSQL connection pool with a maximum size of 10.

If 25 requests simultaneously need a connection, at most 10 can hold connections at once. The remaining requests may wait for a connection, depending on the pool implementation and configuration.

The important point is that 25 active HTTP requests do not necessarily mean 25 active SQL queries. Some requests may be doing other work or waiting for a connection.

Increasing the pool size is not always the solution. The database itself may become overloaded if too many concurrent queries compete for CPU, memory, locks, or disk I/O.

Before changing the pool size, measure connection wait time, query duration, database utilization, and queue depth.

7. How to Interpret the Metrics

Observation	Possible explanation	What to investigate
High latency and high CPU	CPU-intensive work or event-loop blocking	CPU profiles and event-loop delay
High latency and pending DB connections	Connection pool saturation or slow queries	Pool wait time and query duration
High latency and low CPU	Waiting on the database, network, or another service	Distributed traces and dependency latency
Rising active requests and falling throughput	Requests accumulating or resources becoming saturated	Queues, timeouts, CPU, and database health
High error rate	Application failures or dependency problems	Error logs and traces

Do not diagnose a bottleneck from a single metric. Correlate several metrics over the same time window.

8. A Practical Performance Testing Strategy

Before deploying an API, test it under controlled load.

1. Establish a baseline with a small amount of traffic.

2. Increase request rate gradually.

3. Measure throughput, p50, p95, and p99 latency.

4. Monitor CPU, memory, database pool usage, and errors.

5. Identify where latency begins increasing sharply.

6. Inspect the bottleneck before adding resources.

7. Repeat the test after making a change.

For example, a load test might reveal that an endpoint performs an inefficient SQL query. Adding an appropriate index could reduce query duration and improve capacity more effectively than simply increasing the number of application servers.

9. Important Lessons

• Active request count measures requests in progress, not requests completed over time.

• Little's Law relates average concurrency, throughput, and time in the system.

• L / W is not automatically the server's maximum capacity.

• Measure real request and query durations instead of assuming them.

• Database connection pool limits and database processing capacity are different constraints.

• Use controlled load tests to estimate sustainable capacity.

• Use multiple metrics to identify the real bottleneck.

Final principle: Measure first, diagnose the bottleneck, make one targeted improvement, and measure again.