Neuro-cognitive models of word learning propose a role for sleep in consolidating new words, yet evidence for sleep-associated memory benefits outside of experimental contexts is scarce. The present study compared wake- and sleep-associated memory changes in data from Memrise, a publicly available language-learning app. Memory for foreign words and phrases remained very high in accuracy across the 7-12 hour delay, and there were no differences in forgetting between wake and sleep. However, learners were quicker to arrive at the correct translation after a period of sleep compared to wake. This sleep-associated benefit was seen for words but not phrases, supporting a primary benefit for sleep in consolidating new word-forms. As such, we demonstrate that the behavioural benefits of sleep can be observed in real-world language learning, and discuss the promise for combining small-scale lab studies with naturally occurring datasets to understand learning outcomes.