A repeatable method for troubleshooting requests

Every request goes through a predictable sequence of functions. DNS resolves a name to an IP address. The operating system opens a TCP connection (or QUIC connection for HTTP/3). TLS negotiates an encrypted channel. The application protocol (HTTP, SMTP, etc.) sends the actual request. Each step depends on the previous one.

When a request fails, the method is: identify the first function in the chain that did not succeed. Test that function specifically. If it fails, that is where the fault is. If it succeeds, move to the next function. Do not change anything until you have confirmed which function failed.

This sounds obvious. In practice, most failures to diagnose correctly happen because someone skipped to a layer they assumed was the problem, or changed two things at once and lost the ability to know which change made the difference.

Each layer has specific tools that produce specific evidence. Using the wrong tool at the wrong layer produces noise, not signal.

Layers 1 and 2 (Physical and Data Link). Check physical connectivity, interface status, and error counters. On Linux:ip link show shows interface state and error counts.ethtool eth0 shows physical link speed and duplex. On managed switches, check port statistics for CRC errors and input errors.

Layer 3 (Network). Use ping to test basic IP reachability. A successful ping proves the path to the IP address works at Layer 3. A failed ping may indicate routing, firewall blocking ICMP, or the host being down. Use traceroute (or tracerton Windows) to trace the hop path. Traceroute shows where in the path packets stop, but be aware that ICMP rate limiting and asymmetric routing can make the picture incomplete.

Layer 7 (Application), DNS. Use dig domain.comor nslookup domain.com to query DNS resolution. Include a specific resolver to test: dig domain.com @8.8.8.8 compares what Google's resolver returns versus your local resolver. Check for SERVFAIL, NXDOMAIN, or unexpected IP addresses.

Layer 4 (Transport). Use curl -v ortelnet host port to test TCP connectivity to a specific port. A "Connection refused" means the host is reachable but nothing is listening on that port. A timeout means the port is blocked (firewall) or the host is unreachable. A successful connection banner means TCP works; the problem (if any) is at a higher layer.

TLS layer. Use openssl s_client -connect host:443to test TLS. The output shows the certificate chain, handshake result, cipher suite, and any TLS errors. Look for "Verify return code: 0 (ok)" for success and error messages like "certificate verify failed" or "handshake failure" for failures.

Application layer (HTTP). Use curl -v https://host/pathfor a complete end-to-end test. The verbose output shows DNS resolution, TCP connection time, TLS handshake details, HTTP headers, and response code. A 200 means success. 4xx means a client-side request problem. 5xx means the server errored. A redirect loop shows as repeated 3xx responses.

Packet capture. Use tcpdump or Wireshark for full packet visibility when other tools are insufficient. Capture is intrusive and produces large amounts of data. Use it when you need to see exactly what is on the wire, for example to confirm retransmissions, zero-window events, or unexpected RST packets.

Error messages from network tools are precise. "Connection refused" is not the same as "Connection timed out." "SERVFAIL" is not the same as "NXDOMAIN." Reading them accurately is part of the diagnosis.

Connection refused (TCP RST). The destination host is reachable. No service is listening on the target port, or a firewall on the host itself is actively refusing the connection. The host responded with a TCP RST.

Connection timed out. Packets are being dropped without a response. Either a firewall is silently dropping packets, the host is unreachable, or routing is blackholing traffic. No RST was received.

DNS SERVFAIL. The recursive resolver attempted to answer but encountered a failure, often because the authoritative server was unreachable or returned an error.

DNS NXDOMAIN. The authoritative server confirmed the domain does not exist. This is a definitive answer, not a failure to reach the server.

TLS certificate verify failed. The certificate chain could not be validated. Common causes: missing intermediate certificate, expired certificate, hostname mismatch, or untrusted CA.

A diagnosis note has four parts. Symptom: what was observed, by whom, and when. First failed function: which layer failed, with the specific protocol and command output. One observable: the exact command and its relevant output. Next safe test: one specific action that would confirm or narrow the diagnosis.

Example. A developer reports an API endpoint is returning no response.

Symptom: POST https://api.company.com/orders returns no response after 30 seconds from the production application server (10.2.1.5).

First failed function: TCP connection to api.company.com:443 times out from 10.2.1.5. DNS resolves correctly (93.184.216.34). The failure is at Layer 4.

Observable: curl -v --connect-timeout 5 https://api.company.comfrom 10.2.1.5 returns "Connection timed out" after 5 seconds. The same command from the developer's laptop succeeds in 0.3 seconds.

Next safe test: Check firewall rules between VLAN 200 (where 10.2.1.5 lives) and the internet for outbound port 443. A recent change log entry may identify a new rule that dropped this path.

Anyone reading this note knows the symptom, where it fails, the evidence, and what to look at next. No expertise is required. No time is wasted checking the application code.

Escalate when you have confirmed which layer failed but do not have access to fix it. You have verified the fault is in the firewall configuration, but only the network team can change firewall rules. Escalate with the diagnosis note, not with the symptom. "Firewall between VLAN 200 and internet is blocking port 443, confirmed by traceroute and curl timeout" is an escalation. "The internet is down" is not.

Escalate when you have reached the limit of your visibility. You can see that traceroute stops at a specific hop but cannot tell whether that hop is a misconfigured router or a carrier network issue. The diagnosis note records what you can see and where the evidence ends.