The shared vocabulary cache, and why a free tier can survive AI costs
Labelling every unknown word with an LLM is the obvious design and it gets expensive in exactly the wrong direction — the more people read, the more you pay. Here is the cache that fixes it, what it deliberately does not store, and what I do not know yet.
- Naively, every unknown word costs an LLM call, so cost scales with the behaviour you most want to encourage.
- A cache keyed by (language, lemma) means the first person to meet a word pays for it and everyone after gets it free.
- The cache stores linguistic facts only — never your text, never anything identifying you.
- This is a design argument, not a results post. Lemnly is small; the numbers that would prove it out do not exist yet.
The obvious way to build automatic vocabulary labelling is also the one that eventually kills you. Send each unknown word to a model, ask for a translation, a part of speech and an example sentence, store the answer, move on.
It works beautifully in a prototype. Then you notice the shape of the bill.
The cost curve points the wrong way
Imagine a thousand people all read the same Spanish novel. That novel has maybe six thousand distinct lemmas in it. Done naively, that is six million model calls to produce what is, almost every time, the same answer. Hidalgo means the same thing on every one of those thousand phones.
Worse, the cost grows with reading volume — the exact behaviour the product exists to encourage. A user who reads five articles a week costs five times a user who reads one, and the second user is the one I want. Any design where your best users are your most expensive users is a design that will eventually make you do something user-hostile.
If your costs scale with the behaviour you are trying to encourage, you will eventually start discouraging it. Usually by calling it a “fair use limit”.
The fix is unglamorous
One table, keyed by (language, lemma). The first person who needs a translation for a given word in a given language pays for the model call. Everyone after that reads it out of the table in milliseconds, whether they hit it in the same book, a different book or a pasted news article.
Each entry holds:
- The lemma and its language.
- Its most common part of speech.
- A short translation.
- An example sentence.
- An approximate CEFR level.
That last field does more work than it looks like. It is what lets the app skip words far below your level instead of proposing them, which is the difference between an import that hands you thirty useful words and one that hands you four hundred including casa.
What it deliberately does not store
This is the part I would want to know as a user, so it goes above the implementation details rather than below them.
The cache is purely linguistic. This word, in this language, means roughly this. It is a dictionary that gets written incrementally. It contains nothing about who looked a word up, and nothing about what they were reading when they did.
The source text is not kept either. When you import an article the server pulls out the readable text, extracts the lemmas it needs, grabs a surrounding sentence as a candidate example, and discards the rest. If you delete your account, your cards and review history go with it. The cache entries stay, because they were never yours in the first place — they are anonymous linguistic facts, in the same sense that a dictionary entry is.
How a lookup actually goes
- 1Check the cache
A compound index on (language, lemma) makes this a single indexed read. Most words in most texts stop here, which is the entire point.
- 2Check your own history
If the word is already in your deck, it is not proposed again, and in the reader it renders as already-known so your eye skips it.
- 3Only then, the model
Genuinely new words go to the model in a batch, and the answer is written back to the cache so nobody pays for that word again.
The infrastructure is dull, which I consider a good sign. It is one Convex table with an index, batched writes, and the model call behind a server-side action so the API key never goes near a device. Almost all the work was deciding what belongs in the cache. The code took an afternoon.
Versioning, so improvements do not break people
Entries are versioned. When I improve a prompt or move to a better model, the old entries are not invalidated in bulk — a new version gets written when the entry is next touched. Existing users keep the translations they have been reviewing against, new users get the better ones, and nobody wakes up to find a card they have studied for six months now says something subtly different.
That last failure mode is worse than it sounds. A card whose meaning shifts under you does not just confuse you once; it corrupts the scheduling data attached to it.
What I do not know yet
Here is where I have to be straight with you, because an earlier version of this article was not.
The argument above is a design argument. It is not a results report. Lemnly is new and small, and I do not have the user base that would let me publish a meaningful cache hit rate — any number I quoted today would be measuring my own reading habits more than anything else.
What I can say is directional and follows from how language works rather than from my logs: word frequency is brutally skewed, so a small cache covers a large share of running text, and news vocabulary repeats far more than literary vocabulary. Whether that translates into the economics I am betting on is a question I will be able to answer with real numbers in a year, and I will publish them then, including if they are disappointing.
The reason I care about this beyond the bill: the cache is what lets the free tier be an actual free tier rather than a trial with a friendly name. Most competitors cap something — saved words per account, translations per day, explanations per day — and every one of those caps exists because someone’s marginal cost per lookup is not zero.
Getting that marginal cost close to zero is not a clever growth trick. It is just the precondition for not having to put a meter on reading, which is the thing I would most hate to build.
Why write this up at all
Mostly transparency. If you are going to let an app see what you read, you should be able to find out what it does with that text, in plain language, without emailing anyone.
The short version is in the privacy page, and this is the long one. If something here is unclear or you think the design is wrong, tell me — I would rather hear it now than find out at scale.
