Solution Engineer to Lead, WINGS ICT Solutions · 2023 to 2025

Python to Rust: A UDP Ingestion Server That Hit the GIL

An async UDP ingestion server built in Python, then rewritten in Rust when CPU-bound parsing and validation needed a multi-core runtime. Same architecture, new concurrency substrate.

one core to all cores · ~72% less heap traffic / msg · ~1000 concurrent devices

RustTokioUDPJSON Schema

Some details are generalized where a client needs them to be.

Problem

A Python UDP server did its job, but it was one process, and the GIL pinned exactly the expensive work, parsing and validation, to a single core. It could not grow.

Approach

Rewrite it in async Rust. Keep the architecture that already worked, swap the substrate underneath: a socket per core with SO_REUSEPORT, and Arc-shared bytes to stop copying every message.

Result

The Rust design lets the kernel spread packets across workers, and per-message heap traffic is down about 72% by allocation analysis, with the original architecture intact.

About the numbers, and where I’m being careful. One core to all cores: the Python version was a single process, and CPython’s GIL runs the parse and JSON-Schema validation on one core at a time; the Rust version binds a socket per worker with SO_REUSEPORT, so the kernel fans packets out across all of them. The ~72% less heap traffic is an allocation analysis, not a benchmark: I accounted per-message heap traffic from struct sizes (~6 KB down to ~1.7 KB) after moving the hot path to Arc-shared, borrowed data. And ~1000 concurrent devices is the design target, not a measured peak, in-flight work is capped by a semaphore at 512. I never ran a formal load test, which I get to at the end.

This is the most technical of the three case studies. I keep the measurement caveat visible because the lesson is about runtime limits, ownership, and disciplined migration rather than a headline benchmark.

Problem

An IoT product line ships devices that phone home over UDP. Each datagram carries an IMEI, a sequence number, and a sensor payload, and expects a small stateful reply, sampling period, firmware flags, the odd reset bit. The server checks who’s calling, validates the payload against that device’s JSON Schema, forwards it to MQTT, and answers, sometimes only after an MQTT round-trip.

The first version was Python on asyncio, and it had been in production since late 2023. It was well built: a datagram protocol dropped work onto a queue, a pool of worker coroutines drained it, a precompiled regex pulled the IMEI and threw out junk before any JSON parsing, and per-device and global caps shed load under pressure. It ran fine. The problem was structural. Every packet still has to be decoded, parsed, and schema-validated, all CPU work, and in one CPython process the GIL lets only one core do that at a time. Throwing more coroutines at it doesn’t buy you more cores.

Approach

The architecture wasn’t the problem. The runtime was. So I kept the shape and rebuilt what sat underneath it.

Python · asyncio

  • One process, GIL-bound to a single core for parse + validate
  • Queue + N worker coroutines
  • Regex IMEI fast-fail before parsing
  • LRU-cached parsed schemas
  • Per-message decode + json.loads + dict copies

Rust · Tokio

  • SO_REUSEPORT: one socket per worker, kernel spreads across all cores
  • Same regex IMEI fast-fail, proven design kept
  • Arc<[u8]> packets: a “clone” is an atomic increment
  • DashMap lock-free config/schema reads on the hot path
  • Two-level cache: parsed schema + compiled validator

Same pipeline, different engine. The design that already worked in Python carried straight over; the concurrency and memory model got rebuilt in Rust.

  • A socket per core, via SO_REUSEPORT. Instead of one socket feeding a shared queue, each worker binds its own socket to the same port and lets the Linux kernel load-balance datagrams across them. Parsing and validation can now run across workers instead of being concentrated in one Python process, the exact constraint the rewrite was meant to remove.
  • Share bytes, don’t copy them. Packet bytes become an Arc<[u8]> once, and every downstream “copy” after that is just a reference-count bump. Device configs and the (large) JSON schemas live in a DashMap as Arc<T>, so a hot-path lookup hands back an 8-byte pointer instead of deep-cloning a multi-kilobyte struct. Constructors borrow what they read and only clone the fields they actually keep.
  • Same instincts, faster language. The IMEI fast-fail, the periodic config refresh from the platform API, the fire-and-forget device updates, the publish-and-wait over MQTT (a oneshot channel keyed on the response topic), all of it carried over unchanged. It was sound design. It just needed a runtime that could use the whole machine.

Implementation

The Rust service is tokio top to bottom. A config handler polls the platform API on a timer and atomically swaps three DashMap caches, devices, schemas, tenants; readers never block, and reference counting keeps an old config alive for any request still mid-flight across a swap. A semaphore caps in-flight work at 512, so a burst sheds load instead of eating all the memory, UDP is lossy anyway, so dropping under pressure is honest behavior, not a failure. Schemas get compiled once and cached as Arc<Validator>, because compiling them, not running them, is the expensive part.

While I was at it I wrote up the ownership model in detail: which Arc exists to dodge which clone, and where the allocations I couldn’t avoid actually are, serde deserialization, MQTT topic strings, response serialization, around 1.2 KB a message. That accounting is where the ~72% estimate comes from.

Outcome

  • The design can use multiple cores instead of concentrating CPU-bound work in one Python process, which was the entire point.
  • Per-message heap traffic is down ~72% by allocation analysis (~6 KB to ~1.7 KB), from moving the hot path to shared, borrowed data.
  • The Python-first, Rust-for-the-hot-path pattern became my default for ingestion after this, and I used it again on the enforcement pipeline.

What I’d do differently: build the load harness before calling the win done. I can defend the allocation analysis and the concurrency model line by line, but I never stood up a synthetic-traffic generator in front of both versions to get a real throughput and tail-latency curve. Right now the payoff is an argument from architecture and byte-counting. I believe it, but a measured benchmark would have settled it, and that harness would have been tiny next to the rewrite itself.