How to Run a Large Language Model (LLM) on Your Laptop: A Guide from MIT Technology Review
By Grace Huckins | July 17, 2025
In recent years, the power and ubiquity of large language models (LLMs) like ChatGPT have transformed our interaction with artificial intelligence. Now, it’s increasingly possible to run useful LLMs right from the safety and comfort of your own laptop or even smartphone. Here’s a closer look at how you can harness this capability, why it matters, and what tools will help you get started.
A New Era: LLMs on Personal Devices
Simon Willison, a software developer who avidly blogs about local LLMs, has prepared for extreme scenarios by storing open-weight language models on a USB stick. These models, shared publicly by their creators, can be downloaded and run on local hardware without relying on cloud services. Willison imagines using this compact “condensed, faulty version of Wikipedia” as a knowledge base to help reboot civilization if necessary.
But you don’t have to plan for the apocalypse to be interested in running LLMs locally. The community surrounding this practice is vibrant: the subreddit r/LocalLLaMA boasts over half a million members who run and share experiences about LLMs on their personal devices.
Why Run Local LLMs?
Running an LLM locally offers several advantages over relying solely on online models like ChatGPT:
-
Privacy: When you use web-based LLMs, your data often becomes part of model training sets. For example, OpenAI’s and Google’s models use user interactions to improve performance, sometimes requiring tricky opt-out procedures, and recent legal rulings mean user conversations may be retained longer than expected. This raises privacy concerns because personal conversations might end up embedded in the model’s outputs, inadvertently shared with others.
-
Control and Independence: Online AI services frequently update and modify their models behind the scenes. Recent incidents include ChatGPT becoming unusually flattering or other models displaying unexpected, erratic behavior. With your own local LLM, you maintain consistent model behavior and full control—only you can change the software or data it uses.
-
Democratizing AI Power: As Giada Pistilli, principal ethicist at Hugging Face, points out, “Technology means power.” By running LLMs locally, individuals, organizations, and even governments can resist the concentration of AI capabilities in just a few companies, fostering a more open and diverse AI ecosystem.
-
Learning Experience: Smaller local models may have more faults like “hallucinating” (making up information) more often than their larger counterparts, but this helps users understand the strengths and weaknesses of AI more deeply.
As Simon Willison reflects, running local models is “a really good exercise for developing that broader intuition for what these things can do.”
Overcoming the Barrier to Entry
Not long ago, running effective LLMs required expensive server racks costing tens of thousands of dollars. Advances in model efficiency and hardware mean that today, anyone with a decent laptop—or even a smartphone—can run these models.
The rule of thumb is that each billion parameters in the model requires roughly one gigabyte of RAM. For example, a 16 GB laptop can handle models with around 14 billion parameters, assuming minimal background app use.
Getting Started: Tools to Run Local LLMs
You don’t need to be a programmer to explore local LLMs. Depending on your comfort with technology, here are some recommended approaches:
-
Ollama: If you are comfortable with command-line interfaces, Ollama offers a straightforward way to download and run hundreds of models using simple commands.
-
LM Studio: For those wary of coding, LM Studio is a user-friendly app that integrates with Hugging Face’s vast library of models. It provides helpful information on each model’s size, hardware requirements, and user ratings, and includes a chat interface for interaction once a model is loaded.
Running Models on Smartphones
Models can be surprisingly small—and running them on mobile devices is becoming viable. For instance, Meta’s Llama 3.2 with 1 billion parameters can run on an iPhone 12 using an app called LLM Farm. Although smaller models on phones can be prone to generating strange or inaccurate outputs, they are still useful for offline interactions, such as trivia or brainstorming when internet access is unavailable.
Practical Use and Future Potential
Several local LLMs now offer performance sufficient for professional tasks, including journalistic research and writing assistance. While they may not yet fully replace their cloud-based counterparts for heavy-duty workloads, their consistency, privacy, and independence make them enticing alternatives or complementary tools.
As Simon Willison says, “Most people probably don’t need to do this, and that’s fine—but for the people who want to, it’s so much fun.”
Final Thoughts
The era of AI locked behind corporate firewalls is gradually giving way to a more decentralized landscape. By running large language models on your own laptop or phone, you regain privacy, control, and a firsthand understanding of these powerful tools. Whether for curiosity, professional use, or preparation against uncertain futures, local LLMs are becoming an accessible and educational frontier.
For those interested in diving deeper, tools like Ollama and LM Studio are excellent starting points. As the technology evolves, more options will continue to make AI more personal and private.
Grace Huckins is a technology reporter for MIT Technology Review who covers artificial intelligence and emerging technologies.
© 2025 MIT Technology Review. All rights reserved.
