Trevor Gale - Google | LinkedIn (original) (raw)

More often than not, I’m here reflecting: What are the implications of making LLMs endlessly larger? My mind visualizes impossible levels of energy consumption, increasing exponentially each day. It feels like a scene from The Matrix, and not a pretty one. Recently, I’ve been researching about RAG (Retrieval-Augmented Generation), and it sounds promising, there are key points showing potential to reshape how we develop and deploy AI. The aspect that stands out the most to me is the potential to shift away from the “Texas approach” (remember my last post?), RAG could let us move towards smaller(!), modular, specialized models that integrate only the data they need, when they need it, rather than scaling endlessly. It’s like my dream of sibling models working together, creating a more targeted approach for better results. Why am I against ginormous models, you might ask again? LLMs can consume as much energy as powering a small city. City. RAG could allow us to significantly cut energy consumption, by eliminating constant retraining and enormous parameter counts. Have I mentioned I dislike the massive cash burning? Yes, is not my money. Yes, who cares about the overlords money. But honestly, I just hate waste. Did you know the cost of training foundation models have been doubling every 6-10 months? Doubling. Read again. RAG offers the potential to invest once in a base model and then build on it through knowledge retrieval, rather than constant retraining. This approach could make AI access more affordable, opening doors for smaller organizations and underserved communities. Of course, the cynic in me asks: are we simply shifting the computational burden elsewhere? RAG could reduce training costs, but it would introduce database management and retrieval overhead. My cent and a half? What do you think is more costly, monetarily and environmentally, that overhead, or energy consumption? I think you all know my answer. But let’s not forget: RAG doesn’t rely solely on LLM parameters; it retrieves data in real-time from external sources. Meaning APIs, websites, organizational databases, and beyond, and this retrieval approach possess risks and requires robust data security. However, again, the cost will still likely so much lower than today’s soaring energy bills and carbon footprint. So, the question for the 15$ in my wallet is: will AI’s future depend on more power or will be ruled by better processes? My other half cent? A leaner, privacy-conscious, eco-friendlier AI isn’t just a possibility; it should be our goal. All of us. #AI #SustainableTech #EnergyEfficientAI #DataPrivacy