I've been thinking about this for a while too as an FSRS developer [1].
In general, we can think of a spaced repetition system as being (i) Content-aware vs. Content-agnostic and (ii) Deck-aware vs. Deck-agnostic
Content-aware systems care about what you're studying (language, medecine, etc) while Content-agnostic systems don't care about what you're studying.
Deck-aware systems consider each card in the context of the rest of the cards (the "deck") while Deck-agnostic systems consider each card in pure isolation.
Currently, FSRS is both Content-agnostic as well as Deck-agnostic. This makes it extremely easy to integrate into a spaced repetition system, but this also means the model will underfit a bit.
It it interesting to note that you could in practice optimize seperate FSRS models for each deck covering different topics, which would make it Content-aware in a sense. Additionally, "fuzz" is a somewhat Deck-aware feature of the model in that it exists specifically to reduce interactions between other cards in the deck.
Being easy to integrate is an underappreciated feature of FSRS.
Using decks to draw semantic boundaries is likely overly constraining. I think we want to account for finer differences between cards. Decks are coarse and people differ in the ways they use them, some people recommend having just one global deck. Notes are too fine. We explored something in between: a note capturing an idea or concept, plus an associated set of cards. Turns out it's hard to draw idea boundaries. That's why I think it's easier to relate cards by semantic embeddings or more rigid but clearer structures, like the DAG of dependencies suggested elsewhere in this thread.
mabster · 1h ago
On the scheduling end, I'm surprised the article didn't mention https://github.com/fasiha/ebisu which uses Bayesian statistics.
When I was studying Japanese, I was thinking how it's always best to learn words in sentences and that it would be good if the sentences for a particular word were random.
Extending that, the sentences could be picked such that the other words are words scheduled for today meaning much more bang for buck per learning hour.
cosmic_cheese · 43m ago
> When I was studying Japanese, I was thinking how it's always best to learn words in sentences and that it would be good if the sentences for a particular word were random.
>Extending that, the sentences could be picked such that the other words are words scheduled for today meaning much more bang for buck per learning hour.
Just the other day I was thinking about how there’s a good chunk of vocab that could be “mined” from the sentences in my vocab deck.
I think that this idea would work well, but would probably require a whole new SRS program to be able to implement it cleanly. It’s too dynamic for a traditional SRS app like Anki which is pretty static in nature.
ran3000 · 4h ago
I explored memory models for spaced repetition in my master's thesis and later built an SRS product. This post shares my thoughts on content-aware memory models.
I believe this technical shift in how SRS models the student's memory won't just improve scheduling accuracy but, more critically, will unlock better product UX and new types of SRS.
IncreasePosts · 2h ago
I've been playing with something similar, but far less thought out than what you have.
I have a script for it, but am basically waiting until I can run a powerful enough LLM locally to chug through it with good results.
Basically like the knowledge tree you mention towards the end, but attempt to create a knowledge DAG by asking a LLM "does card (A) imply knowledge of card (B) or vice versa". Then, take that DAG and use it to schedule the cards in a breadth first ordering. So, when reviewing a new deck with a lot of new cards, I'll be sure to get questions like "what was the primary cause of the civil war", before I get questions like "who was the Confederate general who fought at bull run"
ran3000 · 2h ago
I'd love to see it.
What I like about your approach is that it circumvents the data problem. You don't need a dataset with review histories and flashcard content in order to train a model.
kebsup · 1h ago
I'm building a SRS language learning app [1] so I've thought about this topic a bit, but I've come to a conclusion that srs algorithms might be just a nerd optimization obsession. My app has "stupid" 1,3,7,15,30 or something like that intervals, and the reality is that if I know a card, I can swipe it within 2 seconds, and if I just barely know it, I can spend 30 seconds on it.
So optimizing the algorithm such that every card comes at the exact right moment might cause all cards to feel too hard or too easy. I think having a mix of difficult and easy cards is actually a feature, not a bug.
This site makes my (more than good) computer's browser crawl to a halt.
jeffalyanak · 1h ago
In the language learning world there are some great tools already for adding content-awareness.
AnkiMorphs[1] will analyze the morphemes in your sentences and, taking into account the interval of each card as a sign of how well you know each one, will re-order your new cards to, ideally, present you with cards that have only one unknown word.
It doesn't do anything to affect the FSRS directly—it only changes the order of new, unlearned cards—but in my experience it's so effective at shrinking the time from new card to stable/mature that I'm not sure how much more it would help to have the FSRS intervals being adjusted in this particular domain.
You mention that FSRS treats each card independently, even if they derive from the same note. I wonder whether you've tried this Anki plugin, which tries to increase the interval between reviews of 'sibling' cards: https://ankiweb.net/shared/info/759844606
ran3000 · 2h ago
Ah, I totally missed this, thanks for sharing it.
Since in Anki the "note" is the editing unit, that works for some cloze deletions but not for QA cards (only for double-sided QA cards). A content-aware memory model would allow you to apply "disperse siblings" to any set of cards, independently of whether they were created together in the same editing interface.
daft_pink · 1h ago
After reading this, I would really like to know what other spaced repetition software there is for things like ai driven speech?
I love Anki and used it before when I needed to memorize things, but would love to know what other options on the market exist.
pessimizer · 2h ago
> [....] Ignoring the following factors means we are leaving useful information on the table:
> 1. The review histories of related cards. Card semantics allow us to identify related cards. This enables memory models to account for the review histories of all relevant cards when estimating a specific card’s retrievability.
> 2. [...]
I've been thinking that card semantics shouldn't be analyzed at all, and just treated as a black box. You can get so much data off of just a few users of a flashcard deck that you could build your own map of the relationships between cards, just by noticing the ones that get failed or pass together over time. Just package that map with the deck and the scheduler might get a lot smarter.
That map could give you good info on which cards were redundant, too.
edit: this may be interesting to someone, but I've also been trying to flesh out a model where agents buy questions from a market, trade questions with each other, and make bets with each other about whether the user will be able to recall the question when asked. Bankrupt agents are replaced by new agents. Every incentive in the system is parameterized by the user's learning requirements.
ran3000 · 2h ago
Yes, that reminds me of knowledge tracing and methods like 1PL-IRT.
I think you can do both and get even better results. The main limitation is that the same flashcards must be studied by multiple students, which doesn't generally apply.
I also love the idea of the market, you could even extend it to evaluate/write high-quality flashcards.
In general, we can think of a spaced repetition system as being (i) Content-aware vs. Content-agnostic and (ii) Deck-aware vs. Deck-agnostic
Content-aware systems care about what you're studying (language, medecine, etc) while Content-agnostic systems don't care about what you're studying.
Deck-aware systems consider each card in the context of the rest of the cards (the "deck") while Deck-agnostic systems consider each card in pure isolation.
Currently, FSRS is both Content-agnostic as well as Deck-agnostic. This makes it extremely easy to integrate into a spaced repetition system, but this also means the model will underfit a bit.
It it interesting to note that you could in practice optimize seperate FSRS models for each deck covering different topics, which would make it Content-aware in a sense. Additionally, "fuzz" is a somewhat Deck-aware feature of the model in that it exists specifically to reduce interactions between other cards in the deck.
[1] https://github.com/open-spaced-repetition/py-fsrs
Using decks to draw semantic boundaries is likely overly constraining. I think we want to account for finer differences between cards. Decks are coarse and people differ in the ways they use them, some people recommend having just one global deck. Notes are too fine. We explored something in between: a note capturing an idea or concept, plus an associated set of cards. Turns out it's hard to draw idea boundaries. That's why I think it's easier to relate cards by semantic embeddings or more rigid but clearer structures, like the DAG of dependencies suggested elsewhere in this thread.
When I was studying Japanese, I was thinking how it's always best to learn words in sentences and that it would be good if the sentences for a particular word were random.
Extending that, the sentences could be picked such that the other words are words scheduled for today meaning much more bang for buck per learning hour.
>Extending that, the sentences could be picked such that the other words are words scheduled for today meaning much more bang for buck per learning hour.
Just the other day I was thinking about how there’s a good chunk of vocab that could be “mined” from the sentences in my vocab deck.
I think that this idea would work well, but would probably require a whole new SRS program to be able to implement it cleanly. It’s too dynamic for a traditional SRS app like Anki which is pretty static in nature.
I believe this technical shift in how SRS models the student's memory won't just improve scheduling accuracy but, more critically, will unlock better product UX and new types of SRS.
I have a script for it, but am basically waiting until I can run a powerful enough LLM locally to chug through it with good results.
Basically like the knowledge tree you mention towards the end, but attempt to create a knowledge DAG by asking a LLM "does card (A) imply knowledge of card (B) or vice versa". Then, take that DAG and use it to schedule the cards in a breadth first ordering. So, when reviewing a new deck with a lot of new cards, I'll be sure to get questions like "what was the primary cause of the civil war", before I get questions like "who was the Confederate general who fought at bull run"
What I like about your approach is that it circumvents the data problem. You don't need a dataset with review histories and flashcard content in order to train a model.
So optimizing the algorithm such that every card comes at the exact right moment might cause all cards to feel too hard or too easy. I think having a mix of difficult and easy cards is actually a feature, not a bug.
[1] https://vocabuo.com
AnkiMorphs[1] will analyze the morphemes in your sentences and, taking into account the interval of each card as a sign of how well you know each one, will re-order your new cards to, ideally, present you with cards that have only one unknown word.
It doesn't do anything to affect the FSRS directly—it only changes the order of new, unlearned cards—but in my experience it's so effective at shrinking the time from new card to stable/mature that I'm not sure how much more it would help to have the FSRS intervals being adjusted in this particular domain.
1: https://mortii.github.io/anki-morphs/intro.html
Since in Anki the "note" is the editing unit, that works for some cloze deletions but not for QA cards (only for double-sided QA cards). A content-aware memory model would allow you to apply "disperse siblings" to any set of cards, independently of whether they were created together in the same editing interface.
I love Anki and used it before when I needed to memorize things, but would love to know what other options on the market exist.
> 1. The review histories of related cards. Card semantics allow us to identify related cards. This enables memory models to account for the review histories of all relevant cards when estimating a specific card’s retrievability.
> 2. [...]
I've been thinking that card semantics shouldn't be analyzed at all, and just treated as a black box. You can get so much data off of just a few users of a flashcard deck that you could build your own map of the relationships between cards, just by noticing the ones that get failed or pass together over time. Just package that map with the deck and the scheduler might get a lot smarter.
That map could give you good info on which cards were redundant, too.
edit: this may be interesting to someone, but I've also been trying to flesh out a model where agents buy questions from a market, trade questions with each other, and make bets with each other about whether the user will be able to recall the question when asked. Bankrupt agents are replaced by new agents. Every incentive in the system is parameterized by the user's learning requirements.
I think you can do both and get even better results. The main limitation is that the same flashcards must be studied by multiple students, which doesn't generally apply.
I also love the idea of the market, you could even extend it to evaluate/write high-quality flashcards.