NanoDB: an embedded document database in Go, with zero dependencies

2026-10-26 · 38 min · Stack Frame

Who this is for: backend engineers who want to see how a database hangs together from WAL to transactions; anyone planning a toy DB or an embedded store; people choosing between SQLite, an LSM key-value store and MongoDB.

TL;DR: NanoDB is a zero-dependency embedded document database in Go — JSON document model, snapshot + WAL persistence, secondary/text/geo indexes, optimistic transactions, atomic batches and a read-only SQL subset, all in 34 Go files: 6,592 lines of source and 3,199 of tests, 9,791 total. It demonstrates how plain every part of a database can be: the WAL is a JSON Lines file, a transaction is a collection snapshot plus a version check, and query planning is "pick the index with the smallest candidate set".

Reading time ~38 minutes · Level: advanced

Contents
  • Preface: why write a database at all
  • Overview: what it actually is
  • Five minutes to first query: three entry points and Go embedding
  • Architecture and code map
  • Data model: documents, collections and _id
  • The persistence core: snapshot + WAL
  • Query execution: the pipeline behind one find
  • The index system: from binary search to bigrams
  • Transactions: collection-granularity optimistic serialization
  • Atomic batches: one WAL record for a whole batch
  • Read-only SQL and the aggregation pipeline
  • HTTP service: tokens, roles and transaction sessions
  • Measured performance
  • Tests and engineering quality
  • Where it sits among the alternatives
  • Known limits (an honest list)
  • When to use it, when not to

Preface: why write a database at all

SQLite is two million lines, LevelDB is a compact LSM textbook, MongoDB is a distributed beast. Writing "a database" in 2026 sounds like reinventing a wheel.

But the point of building a wheel was never the wheel. NanoDB answers a very specific set of questions: what is the shortest path to a database that supports transactions? After stripping all the jargon, how many lines does "committed data survives a crash" actually take? And how do a document model, secondary indexes and optimistic concurrency control mesh on one in-memory data structure?

The answer: about ten thousand lines of Go, zero third-party dependencies, one go build. More importantly, each part can be read by one person in an hour — essentially impossible with real databases. How long does SQLite's pager layer take to read? RocksDB's compaction?

So read this two ways: as a usable tool, and as an anatomy specimen. The first tells you when to use it; the second tells you, if you write your own, what to build first, what to defer, and what must be right the first time.

Overview

NanoDB is a lightweight embedded document database (NoSQL) in Go, currently 0.1.0. "Embedded" means no daemon protocol: it is a library in your process, with a CLI, an interactive shell and a REST service wrapped around it.

DimensionFact
LanguageGo 1.22+, single module nanodb
Third-party dependencies0 (standard library only, no cgo)
Size34 .go files, 9,791 lines (6,592 source + 3,199 tests)
Data modelCollection → JSON document; _id is a non-empty string
Persistencesnapshot.json (full) + wal.log (incremental, JSON Lines)
ConcurrencySafe across goroutines in one process; OS-level exclusive lock on the data directory
Entry pointsOne-shot CLI, interactive shell, REST service
Tests~80% coverage on engine and server, ~75% on cmd

The trade-off in one sentence: do single-machine, memory-resident, JSON-native well, and explicitly give up distribution. No replicas, no sharding, no cluster; one process per data directory at a time. Those are outside the boundary by design, and the README lists all of them.

One detail that captures the temperament: go build -o bin/nanodb ./cmd/nanodb needs no network, because nothing has to be downloaded. In an era of 300 MB dependency trees, that belongs in the introduction.

Five minutes to first query

One-shot CLI:

go build -o bin/nanodb ./cmd/nanodb

# insert (auto-generated _id)
./bin/nanodb -db ./data put users '{"name":"Alice","age":30}'

# filter + sort + paginate
./bin/nanodb -db ./data find users '{"age":{"$gte":18}}' -sort age:-1 -limit 10

# update, count, delete
./bin/nanodb -db ./data update users u2 '{"$inc":{"age":1}}'
./bin/nanodb -db ./data count users '{"age":{"$gt":20}}'
./bin/nanodb -db ./data delete users u2

Successful commands print indented JSON to stdout; failures exit 1 with an Error: prefix on stderr, so scripts can branch on it.

Interactive shell: run without a subcommand and the prompt appears; inside a transaction it becomes nano(tx)>, and quitting or EOF rolls back any uncommitted transaction:

$ ./bin/nanodb -db ./data
NanoDB 0.1.0 interactive shell. Quote JSON arguments with single quotes.
nano> begin
transaction started
nano(tx)> put users '{"name":"Bob"}'
nano(tx)> commit
transaction committed

REST service:

./bin/nanodb -db ./data serve -addr :8080 -token secret
curl -H 'X-NanoDB-Token: secret' localhost:8080/v1/users

All three share one engine rather than being three implementations: cmd/nanodb/main.go (722 lines) only dispatches arguments, server/server.go (764 lines) only routes and authenticates, and the logic lives in engine/. That is the first architectural decision worth remembering: thin entry points, one engine.

Go embedding is the real usage; the CLI and REST are attachments:

db, err := engine.Open("./data", &engine.Options{
    Sync:             true,        // fsync every write
    CompactThreshold: 4 << 20,     // compact when the WAL exceeds 4 MiB
})
if err != nil {
    panic(err)
}
defer db.Close()

doc, _ := db.Insert("users", map[string]any{"name": "Alice", "age": 30})
fmt.Println(doc["_id"])

docs, _ := db.Find("users", engine.Query{
    Filter: map[string]any{"age": map[string]any{"$gte": 18}},
    Sort:   []engine.SortField{{Field: "age", Desc: true}},
    Limit:  10,
})
fmt.Println(len(docs))

The API is a flat method table — Insert / InsertMany / Get / Find / Count / Replace / Update / UpdateMany / Delete / DeleteMany / Batch / Aggregate / QuerySQL / CreateIndex / DropIndex / Compact / Stats / Close — mirroring the CLI subcommands. Two notes: passing nil (or no options) gives Sync: true, while explicitly passing &engine.Options{} means you chose Sync: false, a small Go zero-value trap the README calls out; and all reads and writes return copies, so mutating a returned value never writes back.

Architecture and code map

NanoDB/
├── cmd/nanodb/main.go        # CLI args, dispatch, shell, serve, user admin (722)
├── server/
│   ├── server.go             # REST routes, transaction sessions (764)
│   ├── auth.go               # auth, roles, collection allowlists
│   └── users.go              # users.json store
├── engine/                   # all core logic
│   ├── db.go                 # storage engine, CRUD, WAL, snapshot, compaction (928)
│   ├── query.go              # filter compilation, sort, Top-N, projection (585)
│   ├── index.go              # single/compound indexes, query planning (804)
│   ├── text.go / geo.go      # inverted text index, geo index
│   ├── sql.go                # read-only SQL: lexer, parser, execution (624)
│   ├── aggregate.go          # aggregation pipeline
│   ├── tx.go                 # explicit transactions
│   ├── batch.go              # atomic batches
│   ├── lock*.go              # directory lock (Windows LockFileEx / Unix flock)
│   └── id.go / copy.go       # _id generation, deep copy
└── docs/transaction-isolation.md

Dependencies go one way: cmd and server depend on engine, and engine depends on nobody. There is no abstract interface forest — collection, index and Tx are concrete types, because there is one implementation and reserving interfaces for a hypothetical second one is over-design.

Data model: documents, collections and _id

data/
├── snapshot.json   # full data and index definitions at a point in time
├── wal.log         # operations since the snapshot (JSON Lines)
├── LOCK            # exclusive process lock (empty file, OS-level)
└── users.json      # optional: HTTP accounts

The hierarchy is collection → document; collections resemble tables without a fixed schema. The rules are few and each matters:

  • A document must be a JSON object; _id must be a non-empty string, generated when absent as a 13-hex-digit millisecond timestamp plus 12 random characters — the time prefix keeps generated ids roughly ordered, so inserts do not turn the index into random noise.
  • Writes deep-copy, so mutating the caller's object afterwards does not affect the database; reads return copies. That rule eliminates a whole class of "shared map mutated elsewhere" concurrency bugs at the price of a copy per operation — worth it for memory-resident data of moderate size.
  • Field names accept dotted paths (profile.city) for filters, sorts, projections and indexes, while documents stay freely nested.
  • Field names beginning with $ are operators — a MongoDB-compatibility constraint the README warns about.
  • Numbers written through the CLI or HTTP decode to float64; numbers passed through the Go API keep their type, and comparisons are numeric either way. Cross-entry type behavior is defined explicitly rather than left to be discovered.
JSON typeStoredFilter / sortIndexable
stringYesEquality, regex, lexicographicYes
numberYesNumeric comparisonYes
booleanYesEqualityYes
nullYesOnly $eq: null / $existsExcluded
arrayYesMultikey (any element matches)Per element
objectYesWhole-object equalityExcluded

"Null and objects stay out of indexes" is easy to miss: an index-driven candidate set could then miss documents. NanoDB handles the variants with an incomplete flag plus "the candidate set is only a pre-filter, the full match still runs".

The persistence core: snapshot + WAL

This is the heart of the project. Every database's durability problem reduces to one question: what happens if power is lost mid-write? NanoDB's answer has three layers, none longer than a few dozen lines.

What a record looks like

{"op":"put","coll":"users","doc":{"_id":"u1","name":"Alice","age":30}}
{"op":"delete","coll":"users","id":"u2"}
{"op":"index.create","coll":"users","field":"age","unique":true}
{"op":"batch","ops":[{"op":"put","coll":"users","doc":{"_id":"u3"}}]}
{"op":"drop","coll":"users"}

Seven operation types: put (insert or overwrite; updates are written as overwrite puts), delete, drop, batch, tx, index.create, index.drop. Batches and transactions are a parent record with ops, which is what makes "all or nothing" atomic during replay too.

JSON Lines is a blunt trade: readable, greppable, one record per line, at the cost of space and parse speed versus binary. For a database aimed at moderate data, that is the right exchange — and it reduces crash recovery to "parse line by line, truncate the bad tail".

Layer one: truncating a torn record

If the process dies mid-line, the file ends with half a line. replay returns the length of the valid prefix and openWAL truncates to it:

// replay applies the WAL in order and returns the length of the valid prefix.
func (db *DB) replay(data []byte) (int, error) {
    offset := 0
    for offset < len(data) {
        nl := bytes.IndexByte(data[offset:], '\n')
        if nl < 0 {
            return offset, nil   // no newline: the tail is torn, stop here
        }
        ...
        if err := json.Unmarshal(line, &rec); err != nil {
            return 0, fmt.Errorf("nanodb: corrupt wal near byte %d: %w", offset, err)
        }
        ...
    }
    return offset, nil
}

Two failures are treated differently: no trailing newline is a normal tear, truncate it and keep everything committed before it; a parse failure in the middle is real corruption, and opening fails. Silently skipping a bad line in the middle would disguise data loss as successful recovery — the one boundary recovery logic must never cross.

Layer two: the writer rolls back itself

Recovery alone is not enough; the writer must not leave half a record for the next recovery to find:

n, err := db.wal.Write(data)
if err != nil || n != len(data) {
    // A failed write may leave half a record at the tail; roll back to the
    // previous complete record, or the next append fuses with the fragment
    // into an invalid line that makes the database unopenable after restart.
    db.rollbackWALLocked()
    ...
}

rollbackWALLocked is Truncate(walBytes) + Seek, where walBytes always marks the end of the last complete record — both an in-memory counter and the recovery anchor. The same rollback hangs off fsync failures.

One Windows trap lives in openWAL and deserves quoting: it deliberately does not use O_APPEND, because that flag removes GENERIC_WRITE on Windows and makes later Truncate fail with access denied. Writes are serialized by db.mu and seek explicitly to the end, which preserves append semantics while keeping truncation available. "Cross-platform" is not a slogan; it is decisions like this one.

Layer three: a generation number closes the compaction window

The dangerous window in any snapshot + WAL design is a crash after the snapshot was renamed but before the WAL was cleared: replay then re-applies records already in the snapshot. Harmless for a plain put, a false conflict for a unique index, and for a delete followed by an insert it can resurrect data.

NanoDB uses a generation counter: the snapshot records gen, compaction increments it, and every WAL record carries the generation it was written in. Replay rule, one line:

// Records older than the snapshot generation are already covered; skip them.
if rec.Gen < db.walGen {
    offset += nl + 1
    continue
}

That closes the window entirely: whichever step compaction crashed on, either old-generation records are skipped or the snapshot is still the old generation and replays normally — both states are consistent. A tiny mechanism (one uint64 and a comparison) that draws the line between designed correctness and "it happened not to break".

Compaction order also matters: write a temp snapshot → fsync → atomic rename → then clear the WAL, so a complete old or new snapshot always exists on disk. maybeCompactLocked is called from points where the in-memory state is fully applied (transaction commit, batch completion), because compacting inside appendRecord could snapshot before a pending delete or index creation.

Directory lock: making single-process an OS fact

"One process per data directory" as a convention eventually fails. NanoDB makes it physical with an OS file lock: lock_windows.go uses LockFileEx, lock_unix.go uses flock; a second process gets ErrLocked, and a crashed process releases the lock automatically, so no zombie lock survives. A few dozen lines each behind build tags — the textbook Go approach.

Query execution: the pipeline behind one find

db.Find(coll, q) runs through queryCollection in five steps, each with a clear motive:

planCandidates (pick an index)
    → compileFilter (precompile predicates)
    → match per document
    → Top-N / full sort
    → skip/limit → projection copy

Step one: pick the index with the smallest candidate set

// planCandidates picks the index with the smallest candidate set from the
// filter; nil means a full scan.
func (c *collection) planCandidates(filter map[string]any) map[string]struct{} {
    ...
    conds := make(map[string][]any, len(filter))
    collectConditions(filter, conds)
    ...
    consider := func(ids map[string]struct{}) {
        if ids == nil { return }
        if best == nil || len(ids) < len(best) { best = ids }
    }
    for _, ix := range c.indexes { consider(ix.lookup(conds)) }
    for _, tx := range c.texts    { consider(tx.lookup(conds["$text"])) }
    for field, gx := range c.geos { consider(gx.lookup(conds[field])) }
    return best
}

No cost model, no statistics, no histograms — just "ask every usable index, take the smallest". For a single-machine in-memory database that is the best-value planner: an index lookup is O(log n + hits), so asking three or four is far cheaper than choosing badly and scanning.

The collectConditions comment is worth quoting: it collects top-level and $and conditions, while $or and $not cannot safely produce a candidate set. The union of branches would miss documents if narrowed, so it declines to use an index rather than risk it. The most important quality of a planner is knowing when not to be clever — and the candidate set is only a pre-filter; the full match still runs.

Step two: compile the filter once

A naive implementation re-walks the filter map and recompiles regexes per document. compileFilter hoists that above the scan: it turns the map into a slice (no map iteration per document) and precompiles regexes, with an invalid regex compiled to re = nil meaning "never matches" rather than one error per document. On a 100k-document scan those two decisions are the difference between 2,012 allocations and hundreds of thousands.

Step four: sorting with a limit goes through a Top-N heap

if len(q.Sort) > 0 {
    if q.Limit > 0 {
        k := len(out)
        if q.Skip < len(out) && q.Limit < len(out)-q.Skip {
            k = q.Skip + q.Limit
        }
        out = topNSortDocs(out, q.Sort, k)   // max-heap of size k
    } else {
        sortDocs(out, q.Sort)
    }
}

Asking for the first fifty means scanning once with a heap of size skip + limit, whose top is always the worst of the current best k, then sorting k elements. Complexity drops from O(n log n) to O(n log k): measured on 100k randomly ordered documents, 6.5 ms versus 85.3 ms — thirteen times faster. Sort keys are extracted once up front so comparisons never re-walk nested fields.

The index system: from binary search to bigrams

Secondary indexes and the incomplete flag

An index is an ordered key → document-id map, supporting single fields, compound keys ("a,b"), uniqueness and array multikey (each element indexed; any element matching counts). Range queries binary-search the boundaries, giving O(log n + hits) — the source of "range query 3.1 ms indexed versus 27.3 ms scanned".

Compound indexes have an honest design: documents containing an array field cannot enter a compound index under tuple semantics, so the index is marked incomplete and lookup returns nil (unusable), falling back to a scan. Slow is better than wrong — a partially visible index treated as complete silently drops documents, which is far worse than one order of magnitude of latency.

Text index: Chinese search without a dictionary

The full-text index covers every string in a document and is queried with $text. Tokenization is about thirty lines: alphanumeric runs become lowercase words; runs of Han characters additionally emit bigrams, so 数据库 produces 数据 and 据库. Queries are sliced the same way and ANDed, which supports Chinese without any segmentation dictionary.

The limits are stated plainly: no stop words, stemming or synonyms, no relevance scoring, multi-term queries are pure AND and results are not ranked. It is usable retrieval, not a search engine — and for an embedded database that is the right scope, since most cases only need "fetch the documents containing these words".

Geo index: longitude-ordered plus haversine

The geo index stores entries ordered by longitude and supports $geoWithin with $box and metric $circle. Circles use haversine distance; boxes narrow candidates through the ordered structure first, so expensive trigonometry only runs on a small set. No polygons or GeoJSON geometry, no handling of the 180° meridian, and $geoWithin results are not distance-ordered.

Transactions: collection-granularity optimistic serialization

No lock waits, no deadlocks, no MVCC undo chains — the plainest scheme that fits "in memory, one process": collection snapshots plus version checks.

type Tx struct {
    mu       sync.Mutex
    db       *DB
    snap     map[string]*collection // per-collection snapshot
    versions map[string]uint64      // version when first accessed
    writes   map[string]bool        // collections written in this tx
    walOps   []walRecord
    done     bool
}
  1. Begin takes no database lock — it just starts.
  2. On first access to a collection, under db.mu.RLock, it records the version and deep-copies the whole collection. All later reads and writes use that snapshot: repeatable read for free, and you see your own uncommitted writes.
  3. Commit takes the write lock once and validates the entire read set:
// Validate the whole read set to avoid lost updates and non-repeatable reads;
// read-only transactions validate snapshot consistency the same way.
for name, ver := range tx.versions {
    if tx.db.versions[name] != ver {
        tx.finishLocked()
        return fmt.Errorf("%w: collection %q changed during transaction", ErrConflict, name)
    }
}

On success, the whole change set becomes one WAL record ({"op":"tx","ops":[...]}), the affected collections are replaced, and their versions increment. On failure the caller gets ErrConflict, all changes are discarded, and it retries from Begin.

What that means in practice:

  • Conflict granularity is the collection. If anyone touched any collection you read — even a different document — the commit fails. That is the acknowledged false conflict. A uint64 comparison is far cheaper than per-document tracking, right for low-conflict workloads and prone to wasted retries on hot collections.
  • No deadlocks. The only synchronisation point is the brief write lock at commit; conflicts fail fast instead of queueing, and the database is fully writable between Begin and Commit.
  • Read-only transactions must still Commit(), because that is what certifies a consistent cross-collection view.
  • Transaction methods are goroutine-safe behind tx.mu; after the end every method returns ErrTxDone, and defer tx.Rollback() is idempotent.

The canonical transfer example: Begin(), defer tx.Rollback(), two $inc updates on accounts, then Commit(); on ErrConflict the caller retries. Full isolation levels and retry advice live in docs/transaction-isolation.md — keeping transaction semantics in a document rather than scattering them through comments says something about the project's documentation quality.

Atomic batches: one WAL record for the whole batch

Transactions solve "multi-step and reversible"; batches solve "all at once, no intermediate state". The strategy is clone and stage:

  • Every write runs on clones of the affected collections first; any failure (duplicate _id, unique index conflict, invalid update expression, missing target) discards the batch with the real data untouched.
  • On success all changes become one batch parent record — a single WAL entry, replayed atomically, so there is no half-commit.
  • Batches span collections; update on a missing target is an error while delete on a missing target is not (deleted: false) — both directions defined by what the caller most likely meant.

Cloning costs peak memory during the batch and buys atomicity without rollback logic: a failed path throws the clone away, with no "undo the half-applied index maintenance" code. It shares the snapshot mechanism with transactions.

Batches can also read from stdin (batch -), and because success writes one WAL record, importing 100,000 documents costs one fsync instead of 100,000 — batching pays multiplicatively under a WAL.

Read-only SQL and the aggregation pipeline

engine/sql.go (624 lines) is a hand-written read-only SQL subset: lex → parse → execute, supporting SELECT ... FROM ... WHERE ... ORDER BY ... LIMIT/OFFSET and COUNT(*). Crucially it brings no engine of its own: the parse result becomes the same Query structure the REST and CLI paths use, so it runs through the same compileFilter, the same index planning and the same Top-N sort. "SQL queries perform like find queries" is not a coincidence — it is one pipeline with two entry points.

The boundary is explicit: no JOIN, GROUP BY, HAVING, DISTINCT, subqueries, UNION or column aliases; aggregation is only COUNT(*); LIKE is case-sensitive. For query sugar on an embedded store, that covers most of "pull data out of a script".

The aggregation pipeline has eight stages — $match, $group, $sort, $skip, $limit, $project, $unwind, $count — with accumulators $sum/$avg/$min/$max/$first/$last/$push, named to match MongoDB. Limits: no $lookup, $facet, window functions or $out/$merge, and aggregation does not use indexes.

HTTP service: tokens, roles and transaction sessions

  • Routes: /healthz, CRUD under /v1/<collection> with filter/sort/limit/skip/fields/count query parameters, and admin endpoints such as /v1/admin/compact.
  • Auth: an empty -token disables auth; otherwise requests carry X-NanoDB-Token. The shared token is admin, and per-user tokens can be limited to admin/write/read with exact-match collection allowlists.
  • HTTP transactions: POST /v1/transactions opens one and returns tx_id, operations run under /v1/transactions/$TX/..., then commit or rollback. Sessions live in server memory, roll back after 30 idle minutes, and all uncommitted transactions roll back at shutdown, so long transactions cannot pin snapshot memory forever.
  • Defensive details: 8 MiB request body cap (use CLI batch - for big imports), parameter validation, consistent error codes.

HTTP transactions share the same Tx implementation as Go transactions; the session layer only maps tx_id → *Tx and reaps idle ones — thin entry points, one engine, again.

Measured performance

100,000 documents, 1,000 matches (Windows, local; numbers vary with machine and load):

ScenarioTimeAllocs/op
Equality · indexed3.0 ms2,037
Equality · full scan12.0 ms2,012
Range · indexed3.1 ms2,037
Range · full scan27.3 ms2,012
Projection · indexed3.1 ms2,037
Sort + pagination (10k hits, skip 500 / limit 50)12.9 ms208
Random-order Top-50 (100k hits)6.5 ms6
Random-order full sort (100k hits)85.3 ms6

Four readings: indexes are worth 4–9×, more on ranges because a scan recompares per document while an index binary-searches and walks the hit span; Top-N is worth thirteen times, so never fully sort when you only need the first N; allocs/op is more trustworthy than wall time — consecutive benchmark runs trigger CPU throttling and absolute times can swing over 30%, so compare allocations; and sort + pagination allocates only 208 times because key extraction plus the heap decouple comparison from field access.

The tuning advice is just those mechanisms again: index hot filter fields but not too many; project with fields when you need a subset; always pair sort with limit; trade durability for throughput with -no-sync if you can lose recent writes; raise CompactThreshold for write-heavy workloads, at the cost of longer replays.

Tests and engineering quality

go vet ./...
go test ./...
go test -cover ./...
go test -race ./...        # needs CGO and a C compiler; gcc on Windows
go test ./engine -run '^$' -bench Benchmark -benchmem

All three packages pass (cmd 0.87 s, engine 1.08 s, server 1.58 s), with coverage around 80% for engine and server, 75% for cmd. Coverage spans CRUD, query operators, sort/pagination/projection, four index kinds and uniqueness, aggregation, SQL parsing and execution, batch atomicity, transaction isolation and conflict, WAL replay, compaction and the directory lock, behavior after close, and update error paths — plus auth, roles, allowlists, accounts, routing, 405s, validation, error codes, transaction sessions and timeouts, and the CLI end to end including shell transactions and EOF rollback.

Worth praising: crash and boundary scenarios are part of the regular suite rather than happy-path only — durability incidents live almost entirely there. Benchmarks use 100,000 documents, with BenchmarkSortRandom* doing the A/B between full sort and Top-N that backs rows seven and eight above.

The git history also shows the process being maintained: semantic version tags and releases on every commit, normalized LF endings, explicit Go module cache handling — a 0.1.0 project that built the release pipeline first intends to stay.

Where it sits among the alternatives

DimensionNanoDBSQLiteLevelDB / BadgerDBMongoDB
ShapeEmbedded, single processEmbedded, single processEmbedded KVService, distributed
ModelJSON documents + collectionsRelational + SQLByte KVJSON documents
Query languageMongo-like + read-only SQL + aggregationFull SQLNone (keys/iterators)Mongo queries + aggregation
TransactionsExplicit, collection-granularity optimisticSingle-database ACIDNone / limited MVCC in BadgerMulti-document (4.0+)
Secondary indexesYes, including text and geoB-treesBuild your ownYes, many kinds
PersistenceSnapshot + JSON Lines WALPager + WALLSM + WALWiredTiger + WAL
Dependencies00 (C)0Server + driver
Data sizeFits in memoryTB on diskTB on diskCluster scale

The essence of the differences: against SQLite, SQLite is relational with full SQL and disk paging and can exceed memory, while NanoDB is a document model that lives in memory, selling native JSON and friction-free Go embedding; against LevelDB-family stores, those are key-value engines without a query language or index abstraction, and NanoDB sells the entire layer above them; against MongoDB, Mongo is a network protocol with replica sets and sharding and a completely different operational cost, while NanoDB has no distributed ambition at all.

Known limits (an honest list)

  • Memory resident: documents and indexes are in memory, so size is bounded by RAM; the WAL is JSON Lines and costs more space than a binary format.
  • No distribution: no replicas, sharding or clustering; one process per directory.
  • Transaction boundaries: isolation is fixed, conflicts are collection-granular (false conflicts happen), and there are no savepoints, in-transaction DDL (drop, index create/drop, compact) or cross-process transactions.
  • Text retrieval is approximate: bigrams, no stop words or stemming or synonyms, no scoring, multi-term AND, fixed index field *.
  • Geo is limited: only $box and $circle, no polygons or GeoJSON, no 180° meridian, not distance-ordered.
  • SQL is a read-only subset: no JOIN, GROUP BY, HAVING, DISTINCT, subqueries or UNION; aggregation only COUNT(*).
  • Aggregation skips indexes: no $lookup, $facet, window functions or $out/$merge.
  • Compound indexes fall back on arrays: documents with array fields never enter them, so planning falls back to a scan.
  • Coarse permissions: collection allowlists are exact-match, with no wildcards or field-level rights; 8 MiB HTTP body limit.
  • Array POST is not atomic: use a batch or transaction when you need atomicity.

When to use it, when not to

Good fit: local storage inside Go tools and services — configuration, task queues, crawler state, desktop app data — where you do not want to run a database service to store tens of thousands of JSON rows; read-heavy workloads whose data fits in memory, where indexes and Top-N give millisecond responses; teaching and forking, because ten thousand dependency-free lines with every part present are an excellent starting point for changing the WAL format, adding an index kind or practicing transactions; and edge or single-machine deployments where a single binary with no cgo and no external service is the whole story.

Poor fit: data larger than memory or terabyte-scale durability (that is SQLite and LSM territory); multi-process writes to one directory or multi-node high availability (explicitly out of scope); transaction-heavy writes on hot collections (collection-level optimistic conflicts mean many retries); and production needs around relevance-ranked full-text search, complex SQL analytics or precise geospatial computation.

Closing

Back to the three opening questions. How short is the shortest path to a transactional database? A Tx struct plus a version-check loop: engine/tx.go is entirely "collection snapshot + read-set validation at commit", with no lock manager and no undo log, yet it delivers repeatable read and lost-update protection and reports ErrConflict so the caller can retry.

How many lines is "committed data survives a crash"? Three layers: replay finds the valid prefix and truncates a torn record; appendRecord rolls back a failed write to the previous complete record; and the gen counter makes crashes in the compaction window self-resolve. Under a hundred lines of core — the hard part is not the code but thinking through every crash point.

How do the document model, indexes and concurrency mesh? Writes go through collection.add, which maintains all four index kinds synchronously; transactions carry those indexes along inside the cloned snapshot; and planning only picks the smallest candidate set, with the full match as the backstop.

NanoDB will not replace SQLite and has no intention of replacing MongoDB. Its value is laying the black box out on the table: the WAL is newline-delimited JSON, a transaction is a version comparison, and query planning is picking the smallest candidate set. The mechanisms are plain enough to memorize — and plainness is exactly why they can be trusted.

If you have ever thought "I should write a database", the answer here is yes, and it does not take many lines. The next step is not more papers: clone it and break replay, Commit and planCandidates one at a time, then see whether the tests still pass. That is a faster introduction than any article.

Along the same chain: Write the log first: WAL internals and B+ tree page splits in detail.