ANT-2026-QRYCJNAP · dnsmasq

denial-of-service medium

CVE-2026-13002 CVE-2026-4890

Severity Claude high · Security research firm high · Maintainer medium

Discovered by an unreleased Anthropic model

REPORT

Anthropic's analysis, sealed at approval. Disclosure to the maintainer was performed by Trail of Bits.

ANT-2026-QRYCJNAP: DNSSEC NSEC typemap infinite loop hangs daemon

In dnsmasq's DNSSEC validator, prove_non_existence_nsec() (dnssec.c:1340-1356) and the matching NSEC3 loop (1507-1523) iterate NSEC type-bitmap windows but decrement rdlen and advance p by only p[1] (bitmap length) rather than p[1]+2 (header + bitmap). When an attacker supplies a two-byte window block [window#][0x00] with window# != (qtype>>8), the bounds check passes and the loop state never changes, yielding an infinite loop. The attacker controls the NSEC RDATA bytes; a valid RRSIG over malformed bytes is accepted (line 2313 path), and the wildcard path at line 2275 reaches the loop before NSEC signatures are even verified. Because dnsmasq is single-threaded with no per-handler timeout, one hostile reply permanently hangs DNS, DHCP, TFTP, and RA services until the process is killed.

Target

Project: dnsmasq
Version: commit b95d5a25777cc8910d2c2a921c68c778a3b30498 (still present at HEAD)
Location: src/dnssec.c:1354

Technical Details

Built a unit harness that #includes dnssec.c and calls the static prove_non_existence_nsec() with a crafted NSEC RR whose type-bitmap window has window#=1 and bitmap-length=0 (p[1]==0), with the NSEC owner name equal to the queried name so the rc==0 branch is taken. The function never returned; a 3-second SIGALRM watchdog fired, proving the infinite loop. Root cause confirmed at src/dnssec.c:1354-1355: the loop advances by p[1] bytes instead of p[1]+2, so a zero-length bitmap leaves rdlen and p unchanged forever. Identical pattern present in the NSEC3 path. Reaching this in production requires an attacker who controls a DNSSEC-signed zone (or can forge a validly-signed NSEC), which is plausible for any domain owner — a single hostile signed reply hangs the daemon's single-threaded resolver loop.

Crash signature: Hang / 100% CPU in prove_non_existence_nsec typemap loop; 3s SIGALRM watchdog fired (no return)

Reproduction

  1. Register a domain and provision a valid DNSSEC chain (DS in parent, signed DNSKEY).
  2. Configure the authoritative server to answer nx.evil.example AAAA with a NODATA response whose authority section contains an NSEC record owner==qname and type-bitmap RDATA 0xFF 0x00, plus a valid RRSIG over it.
  3. Induce any client behind the target dnsmasq to resolve nx.evil.example (e.g., a webpage embedding the hostname).
  4. dnsmasq forwards, receives the reply, validates the RRSIG, and calls prove_non_existence() at dnssec.c:2313 for the unanswered question.
  5. prove_non_existence_nsec() matches owner==qname (rc==0 at line 1312) and enters the typemap loop with p[0]=0xFF, p[1]=0x00, rdlen=2; the loop never advances.
  6. The single-threaded event loop is blocked; the process spins at 100% CPU until killed.

[No reproducer or sanitizer output attached — request from security-cvd@anthropic.com if needed.]

Acknowledgement

This vulnerability was discovered by Claude, Anthropic's AI assistant, and triaged by the Anthropic security team in collaboration with Anthropic Research. Please direct questions to security-cvd@anthropic.com and reference ANT-2026-QRYCJNAP.

Reference: ANT-2026-QRYCJNAP
Anthropic CVD Policy: https://www.anthropic.com/coordinated-vulnerability-disclosure

SECURITY RESEARCH FIRM ANALYSIS

Triage and disclosure were performed by Trail of Bits.

Verdict
true positive
Severity
high
TIMELINE

Dates from discovery through public reveal.

  1. 2026-04-08 Reported to tracker
  2. 2026-04-21 Maintainer acknowledged
  3. 2026-05-07 Sent to maintainer
  4. 2026-05-09 Patch released
  5. 2026-08-17 Publicly revealed
PROVENANCE

SHA-3-512 hash:

aa75ba64f155bef4fe2bf47909666cea6dbecd39cfa0804362552cd6bb47eab70613db0438ce271d146d51e68e26b89bfb8ec8c1c8caf89093405a2f5e6f8585

Committed 2026-05-07 00:02 PT

Revealed 2026-08-17 10:47 PT

Verify (download preimage.json)

Show preimage JSON 
{
  "ant_id": "ANT-2026-QRYCJNAP",
  "bug_class": "Denial-of-service",
  "claude_severity": "high",
  "commit_sha": null,
  "created_at": "2026-04-09T05:32:53+00:00",
  "description": "In dnsmasq's DNSSEC validator, prove_non_existence_nsec() (dnssec.c:1340-1356) and the matching NSEC3 loop (1507-1523) iterate NSEC type-bitmap windows but decrement rdlen and advance p by only p[1] (bitmap length) rather than p[1]+2 (header + bitmap). When an attacker supplies a two-byte window block [window#][0x00] with window# != (qtype>>8), the bounds check passes and the loop state never changes, yielding an infinite loop. The attacker controls the NSEC RDATA bytes; a valid RRSIG over malformed bytes is accepted (line 2313 path), and the wildcard path at line 2275 reaches the loop before NSEC signatures are even verified. Because dnsmasq is single-threaded with no per-handler timeout, one hostile reply permanently hangs DNS, DHCP, TFTP, and RA services until the process is killed.",
  "discovered_at": null,
  "location": "src/dnssec.c:1354",
  "poc_sha256": null,
  "preimage_version": 1,
  "project": "dnsmasq",
  "reproduction": [
    "1. Register a domain and provision a valid DNSSEC chain (DS in parent, signed DNSKEY).",
    "2. Configure the authoritative server to answer `nx.evil.example AAAA` with a NODATA response whose authority section contains an NSEC record owner==qname and type-bitmap RDATA `0xFF 0x00`, plus a valid RRSIG over it.",
    "3. Induce any client behind the target dnsmasq to resolve `nx.evil.example` (e.g., a webpage embedding the hostname).",
    "4. dnsmasq forwards, receives the reply, validates the RRSIG, and calls prove_non_existence() at dnssec.c:2313 for the unanswered question.",
    "5. prove_non_existence_nsec() matches owner==qname (rc==0 at line 1312) and enters the typemap loop with p[0]=0xFF, p[1]=0x00, rdlen=2; the loop never advances.",
    "6. The single-threaded event loop is blocked; the process spins at 100% CPU until killed."
  ],
  "technical_details": "Built a unit harness that #includes dnssec.c and calls the static prove_non_existence_nsec() with a crafted NSEC RR whose type-bitmap window has window#=1 and bitmap-length=0 (p[1]==0), with the NSEC owner name equal to the queried name so the rc==0 branch is taken. The function never returned; a 3-second SIGALRM watchdog fired, proving the infinite loop. Root cause confirmed at src/dnssec.c:1354-1355: the loop advances by p[1] bytes instead of p[1]+2, so a zero-length bitmap leaves rdlen and p unchanged forever. Identical pattern present in the NSEC3 path. Reaching this in production requires an attacker who controls a DNSSEC-signed zone (or can forge a validly-signed NSEC), which is plausible for any domain owner — a single hostile signed reply hangs the daemon's single-threaded resolver loop.",
  "title": "DNSSEC NSEC typemap infinite loop hangs daemon",
  "vendor_severity": "high"
}