Children and adults benefit from a new word’s phonological neighbours during explicit vocabulary instruction, suggesting that related prior knowledge supports learning across development. This study examined lexical neighbourhood structure during incidental word learning—limiting opportunities for strategically engaging prior knowledge—and tested the hypothesis that prior knowledge would provide additional support during subsequent consolidation. Children (Experiment 1) and adults (Experiments 2, 3) were presented with 15 pseudowords embedded in spoken stories, and then tested on their recognition and recall of the new word-forms immediately, the next day, and one week later. The pseudowords had either no, one, or many English phonological neighbours, varying the potential connections to existing knowledge. After hearing the pseudowords in a spoken story context, children did not benefit from phonological neighbours in recall, and were better at recognising items without neighbours. Adults showed a recall benefit for pseudowords with lexical neighbours when they could read along with the story (Experiment 2), but this benefit was less apparent when written text was removed (Experiment 3). The neighbour influence did not change with opportunities for consolidation in any experiment, nor did it relate to learners’ existing vocabulary ability. Exploratory analyses revealed that children experienced bigger benefits from offline consolidation overall, with adults outperforming children only for many-neighbour items one week after exposure. We discuss how the neighbour benefit in word learning may be constrained by learning context, and how the enhanced benefits of offline consolidation earlier in development extend to vocabulary learning in more naturalistic contexts.