A year ago, many people were excited about using large language models on their own computers, which led to the creation of different tools to make this possible. One tool I tried and remember well is LM Studio. LM Studio is a desktop application that lets you run an open-source language model on your computer, providing a sense of privacy. At first, this sounds appealing, but I soon noticed some major problems with this approach.
First of all, it's very technical. Using tools like LM Studio means you have to choose from many different open-source models and figure out which version works best for you. You also need to stay updated on each model's changes to pick the best one. Even after all this, the user experience isn't great. Running these models on your computer can use a lot of RAM, slowing down other applications. Plus, you might experience delays and get worse results compared to free services like ChatGPT, which offer access to more powerful models. This makes the whole process cumbersome and not worth the effort for most people, especially when free and easy-to-use alternatives are available. Machine learning enthusiasts might find it interesting, but the average user won't bother.
From a user’s perspective, there isn’t much benefit in using a complex solution that’s hard for most people to set up. Even if you do get it working, the experience isn’t as good as the free tools provided by platforms like OpenAI, Cohere, or Anthropic. Most people don’t care enough about the privacy of their questions to a chatbot to go through the trouble of using LM Studio or similar tools.
So, what’s next? I think local large language models (LLMs) still have potential. As technology improves, we might see smaller, yet more powerful language models. I recently wrote an article suggesting that smaller language models could be the future. If we can create small models that perform as well as something like GPT-4, people might be encouraged to use them on their personal computers, as long as the desktop application is easy to use.
The real challenge is figuring out how to make money from this approach. Developing a solid business model for monetizing local LLMs is what will encourage innovation in this area. However, so far, I haven't seen a clear way to make this profitable. If we can solve this problem, we might see a resurgence of interest in local LLMs. Until then, it seems like the idea of running large language models on your computer will remain a niche interest for a small group of enthusiasts.
