Spread-em: designing a distributed file store with AI
Spread-em is a distributed file storage system for serverless social media apps. It spreads content out across multiple mobile devices and servers, rather than a traditional centralized server, and provides indexing and sharing through a combination of torrent-like technologies.
This is a "bit torrent x 5" level of complexity, due to the use of mobile devices, fragmentation of media types, grouping posts and media into distribution lists, and the challenge of providing access through NAT routers.
We were keen to see how well Claude could assist system design, then research, draw in and integrate the broad range of technologies required.
How Spread-em nodes communicate
On a local network, nodes announce their availability over UDP multicast and sync directly over TCP — a fast path that avoids any server round-trip.
Across the internet, nodes discover each other through a distributed hash table and a STUN server, punch through NAT routers with UDP hole punching, and fall back to a relay server with a mailbox for messages when a direct connection can't be established.
The challenge
Using AI to research and implement a multitude of network protocols, punching through firewalls, network-based indexing, proxies and file sharing.
The original effort was too hard to even start — the breadth of unfamiliar protocols made it a daunting greenfield problem.
Developing with AI
- 5 days to 60% product readiness
- Proof of AI capability: 100%
Technology
mDNS, DHT servers, UDP multicast, TCP, UDP hole punching, STUN server, ICE signalling.
AI workflow
The work came down to extensive research tasks paired with careful, progressive prompt engineering — using AI to pull in and integrate a wide range of networking technologies, then steering it incrementally toward a working system.