Large language models (LLMs) are being developed with enormous context windows, potentially impacting the role of retrieval-augmented generation (RAG). The ability of LLMs to handle large amounts of context is being explored through various techniques, including memory sparse attention and context engineering. The implications of these advancements on the development and accessibility of top AI models are being discussed.