go-redis — Redis Server from Scratch

Overview

go-redis is a reimplementation of the Redis server in Go, built to understand how a database that handles hundreds of thousands of operations per second on a *single thread* actually works. It speaks the real RESP wire protocol, so redis-cli connects to it directly with no adapter. The concurrency model, expiry strategy, eviction policy, and persistence mechanism are all modeled on how Redis itself solves these problems.

Core Features

•Single-threaded, non-blocking TCP server built directly on epoll syscalls — one thread multiplexing many clients, with no goroutine-per-connection.
•Hand-written RESP protocol encoder and decoder covering simple strings, errors, integers, bulk strings, and arrays, with command pipelining support and unit tests.
•Commands: PING, SET (with EX), GET, DEL, TTL, EXPIRE, INCR, INFO, CLIENT, LATENCY, and BGWRITEAOF.
•Dual expiry strategy — passive expiry on read, plus an active sampling cron that repeatedly checks 20 random keys and keeps looping while more than 25% of the sample is expired.
•Configurable eviction: simple-first, allkeys-random, and an approximated LRU that samples keys into an eviction pool ranked by idle time rather than maintaining a full LRU list.
•Type and encoding metadata packed into a single byte — 4 bits of type, 4 bits of encoding — with WRONGTYPE assertions on command dispatch.
•AOF persistence that replays the keyspace as SET commands, triggered via BGWRITEAOF.
•Keyspace statistics exposed through INFO across multiple logical databases.

System Architecture

•Server Layer: Both a synchronous, connection-per-client server and an asynchronous epoll event loop are implemented, making the performance difference between the two models directly observable.
•Event Loop: Registers the listening socket with an epoll instance, sets accepted client sockets non-blocking, and dispatches read-ready file descriptors — interleaving a cron tick for active key expiry between EpollWait cycles.
•Protocol Layer: Decodes incoming byte streams into commands and encodes responses back into RESP.
•Core Layer: The keyspace store, object model with type/encoding tags, expiry table, eviction strategies, and AOF dumping.

Key Challenges

•Writing a correct RESP parser against a byte stream where a single read may contain partial or multiple commands.
•Using raw epoll syscalls in Go — non-blocking sockets, edge cases around accepting connections, and cleaning up file descriptors when clients disconnect.
•Fitting periodic maintenance work into an event loop that otherwise blocks indefinitely waiting for I/O.
•Approximating LRU through sampling, since exact LRU would cost more memory and pointer maintenance than the eviction is worth.
•Handling clock wrap-around in the 32-bit access-time counter used for idle-time calculations.

Key Learnings

•Why Redis is single-threaded, and how I/O multiplexing makes that a performance advantage rather than a limitation.
•The tradeoffs behind probabilistic algorithms in databases — sampled expiry and approximated LRU trade exactness for predictable, bounded cost.
•Binary protocol design and streaming parser construction.
•Low-level Linux networking through the syscall package instead of Go's net abstractions.
•Memory layout decisions, such as packing type and encoding into one byte.

Impact

•Built a Redis-compatible server that works with standard Redis clients, covering the networking, protocol, storage, expiry, eviction, and persistence layers.
•Developed a concrete understanding of event-driven server architecture, applicable well beyond Redis itself.