home / ctf / pwnsec

peekaboo

seccomp allows 8-byte writes from one page only, so the ciphertext is found by mmap binary search and the 24-bit srand seed does the rest.

2026.09.19 PwnSec CTF 2026 Pwn
FLAG pwnsec{c97e423274716aa2}

I hid something in this process. You can run any code you want inside it. You just can’t look at anything. — @daeseong

Summary

The process runs our shellcode, but seccomp allows write() only from one page (0x31339000) and only 8 bytes at a time. The flag is encrypted with AES-256-GCM-SIV and left, base64-encoded, on a page at a random address — but the key comes from rand(), and the srand() seed is built from just 3 bytes (24 bits) of /dev/urandom.

The attack has two halves:

  1. Shellcode (online) — binary-search for the ciphertext page with mmap(MAP_FIXED_NOREPLACE) probing, copy it into the permitted page, and leak it 8 bytes at a time.
  2. Brute force (offline) — recover the key from the leaked [nonce][tag][ct] by trying all 2^24 seeds, then decrypt.

Binary analysis

prob is x86-64 PIE, stripped, glibc, linked against libseccomp and libcrypto (OpenSSL 3). main does the following (all recovered with objdump/capstone):

1) A 24-bit seed

fd = open("/dev/urandom"); read(fd, &seed, 3); close(fd);   // only 3 bytes!
srand(seed);                                                // seed in [0, 0xFFFFFF]

seed is a 4-byte integer but only three bytes are filled; the top byte stays 0 (bss) → 24 bits of entropy.

2) The ciphertext page address also comes from rand()

r1 = rand();
region_D = (0x100000000 + (r1 & 0x3fffff)) << 12;   // one page in [0x100000000000, 0x1003fffff000]

An RW page is placed at that random address with mmap(MAP_FIXED). The other three pages are at fixed addresses: 0x31337000 (RWX, shellcode), 0x31338000 (RW, stack) and 0x31339000 (RW, the write-permitted page).

3) The key is 32 bytes of rand()

for (i = 0; i < 32; i++) key[i] = rand() & 0xff;    // low byte of rand() #2..#33

4) Flag encryption — and the trap

read(open("./flag"), region_D, 0x100);              // flag into region_D
RAND_bytes(nonce, 12);                              // a genuinely random OpenSSL nonce

out = malloc(flaglen + 0x1c);
memcpy(out, nonce, 12);                             // out[0:12] = nonce

EVP_EncryptInit_ex2(ctx, aes256gcmsiv, key, NULL, NULL);   // * IV = NULL
EVP_EncryptUpdate(ctx, NULL, &l, nonce, 12);               // * the nonce is used only as AAD
EVP_EncryptUpdate(ctx, out+0x1c, &l, region_D, flaglen);   // out[28:] = ciphertext
EVP_EncryptFinal_ex(...);
EVP_CIPHER_CTX_ctrl(GET_TAG, 16, out+0x0c);                // out[12:28] = tag

// out = [nonce(12)][tag(16)][ct(N)] -> base64 -> stored back into region_D

The critical detail: the IV argument to EVP_EncryptInit_ex2 is NULL. So OpenSSL’s GCM-SIV uses a zero IV, and the 12 random nonce bytes only enter authentication as AAD. (Reading it as IV=nonce at first meant no seed would ever decrypt; only spotting xor ecx,ecx — IV=NULL — in the disassembly fixed it.)

5) The sandbox

read(0, 0x31337000+0x9a, 0xf65);       // our shellcode, from stdin, in a single read
setrlimit(RLIMIT_CPU, {2,4});          // 2 seconds of CPU
setitimer(ITIMER_REAL, {10s});         // 10-second alarm
prctl(PR_SET_NO_NEW_PRIVS, 1);

seccomp(default=KILL):
    ALLOW mmap, munmap, exit_group
    ALLOW write IFF arg2(count) <= 8 AND (arg1(buf) & ~0xfff) == 0x31339000

// jump to our shellcode via a trampoline (all registers zeroed, rsp=0x31338000)

Even read is denied. write works only from the 0x31339000 page and only in 8-byte chunks. The ciphertext is at a random address and we do not know it — hence “you can run code but you cannot look at anything”.

Exploit

Part 1: shellcode (sc.asm, 184 bytes)

Locating the ciphertext page by mmap binary search. The possible addresses are (0x100000000+k) << 12 for k in [0,0x3fffff] — 4 M pages across the window [0x100000000000, 0x100400000000). mmap(addr, len, PROT_NONE, MAP_FIXED_NOREPLACE|...) fails with EEXIST if the range overlaps an existing mapping, and 22 probes on that signal pin the page down:

lo=0x100000000000, hi=0x100400000000
while hi-lo > 0x1000:
    mid = lo + ((hi-lo)/2 & ~0xfff)
    if mmap([lo,mid), FIXED_NOREPLACE) == lo:   # no clash -> target is in [mid,hi)
        munmap([lo,mid)); lo = mid
    else:                                       # EEXIST -> target is in [lo,mid)
        hi = mid
# lo == region_D

Then memcpy the bytes of region_D into the permitted page 0x31339000 and leak the base64 string with repeated write(1, 0x31339000+off, 8), finishing with exit_group.

Part 2: offline brute force (brute.c)

Reimplement glibc’s rand() (the TYPE_3 additive generator, verified against real glibc output) and sweep all 2^24 seeds across 8 threads:

for seed in 0..0xFFFFFF:
    srand(seed); rand();                 // rand()#1 = the address, discarded
    for i in 0..31: key[i]=rand()&0xff

    // exactly as the binary does it: IV=NULL, AAD=nonce
    DecryptInit(key, iv=NULL); set_tag(tag); AAD(nonce,12); Decrypt(ct)
    if DecryptFinal() == 1: print flag    // tag verifies = correct key

With OpenSSL 3 (hardware AES) that is ~20 seconds worst case. The tag matched at seed 0x5f83f8.

$ python3 leak.py 3551a951ff111c77.chal.ctf.ae 443
[*] received 2048 bytes
vdG2+D2AhoPx2KzcubpkV6uE1pycNPEPTeTk1sTNDEz8Vggefhr51YJaOZ/oii+VpTdRkQ==

$ ./brute "vdG2+D2A...RkQ=="
[*] decoded 52 bytes (nonce12 + tag16 + ct24)
[+] seed=0x5f83f8 FLAG: pwnsec{c97e423274716aa2}

Notes

No local testing. On Apple Silicon, qemu-user cannot handle this binary’s seccomp syscall and dies during setup (exit 255). So the crypto and brute force pipeline was verified natively end to end, while the shellcode was reviewed carefully and then run against the real remote (native x86_64).

The biggest time sink was IV=NULL versus IV=nonce. A self-consistent test harness passed happily, but it did not match the binary, so the real blob decrypted under no seed at all. The lesson: a verification harness has to replicate the target’s actual call sequence. One line of disassembly — xor ecx,ecx (IV NULL) — was the answer.

The design weakness is not the seccomp filter, it is key entropy. seccomp and the randomised address only make peeking hard; a 24-bit seed undoes everything. However thoroughly you block observation, once the ciphertext leaks the key falls offline.

Files

  • sc.asm, sc.bin — the leaking shellcode (nasm)
  • leak.py — sends the shellcode over TLS and captures the base64 leak (python3 leak.py <host> [port])
  • brute.c, brute — glibc rand reimplementation plus the 2^24-seed brute-force decrypt
  • src/ — challenge binary and libraries (prob, lib/libcrypto.so.3)
  • peekaboo.zip — original handout (password: infected)
#seccomp#shellcode#aes-gcm-siv#brute-force