Not generally opposing large language models and some of the use cases they cover, but, talking about hosting these things locally, it's again and again incredibly amazing to see the hardware thrown at this in order to work. Like, adding some sort of GPU that amounts to close to at least €8k and /still/ knowing this device, worst case, might be blocked for several seconds if one user tries to get an answer to a more complex question? And scaling such a system, locally, for companies with a five-digit user base? Having this stuff on ones budget leaves one more than just slightly dizzy. Consuming cloud resources hides some of these direct costs but of course they're there, in terms of hardware needed and in terms of energy spent on that. Plus ... we're not even talking about dependency and entry barriers here for using technology. There's no real issue in getting commodity computer hardware to run meaningful applications these days. But at this very point, hardware investments for most of these kinds of technologies are a pain to deal with it if wanting to get anywhere near serious.
