A large language model (LLM) is a type of AI model trained on very large amounts of text to understand and generate human language. Given a prompt, it predicts the most likely next pieces of text, one after another, which lets it answer questions, summarize documents, draft content, translate and write code. LLMs are the engine behind today’s AI assistants and copilots and many chatbots, and the best-known form of generative AI. On its own an LLM is a capability, not a business solution; it becomes useful when built into an application with data, controls and a defined task.
At a glance
- An LLM is a model trained with machine learning on large volumes of text; it generates language by predicting what comes next.
- Well-known model families include GPT, Claude, Gemini and Llama; many buyers use them through applications rather than directly.
- Outputs are probabilistic: the same prompt can give different answers, and fluent answers can be wrong.
- A model reflects only its training data, which may be dated or wrong, plus whatever you supply in the prompt or through connected data; it doesn’t know your current or internal facts unless the application provides them.
- Data handling, model choice and pricing are set by the application’s and provider’s terms, which vary widely.
What problem it solves
Much business work runs on language: emails, contracts, tickets, policies, reports, code and conversations with customers. Traditional software could store and search that text, but it could not read it, summarize it or write new text without explicit rules for every case.
LLMs handle language tasks that used to need a person. They can summarize a long document, draft a reply, answer a question from a set of policies, classify incoming requests, extract terms from contracts or explain a block of code. For many teams that means less time on routine reading and writing, faster access to internal knowledge and new ways for customers to get help. The catch is that the output needs checking in proportion to the stakes.
How it works
Training. The model is trained on a very large body of text, learning statistical relationships between words, phrases and ideas. It is then usually tuned further with examples and human feedback so it follows instructions and declines some requests. Training happens before you use it; the model doesn’t change as you use it.
Prompts and context. When you send a prompt, the model processes it along with any extra material supplied in the same request, such as a document, chat history or search results, up to a limit called the context window. That supplied material is how an application gives the model your information.
Generation. The model produces its response piece by piece, called tokens, each time choosing a likely next token given everything so far. Settings can make outputs more varied or more consistent, but not guaranteed to be correct.
Grounding and tools. Applications often use retrieval-augmented generation (RAG) to look up relevant internal content and pass it to the model, and connect it to other systems through tool interfaces such as the Model Context Protocol (MCP). Agentic AI goes further, letting the model plan and take actions.
Model versus application. The model is the engine. The application, such as an office copilot, a CRM assistant or an internal chatbot, adds the interface, data connections, permissions, logging and contract terms. That is what many organizations actually buy, and it is why two products using the same model can behave very differently. Providers update and retire models over time, which can change behavior and cost.
When it matters for buyers
- Evaluating AI features in software you already use. Ask which model is underneath, whether you can choose it and what happens when it is updated or retired.
- Choosing between public tools and enterprise versions. Data retention, training use and admin controls often differ between consumer and business tiers.
- Connecting LLMs to internal data. Permissions must carry through, or the assistant may show people content they shouldn’t see; prompt injection is a related risk.
- Regulated or high-stakes uses. Legal, financial, health and customer-facing outputs need defined human review and records of what was generated.
- Self-hosting or private deployment. Running open-weight models in your own environment, sometimes called private AI, adds control and infrastructure cost.
Questions to ask vendors
- Which models does your product use, who hosts them, and can we choose or pin a model version?
- Are our prompts, files and outputs used to train any model, and how long are they retained?
- Where is our data processed and stored, and who at your company or your model provider can access it?
- How does the product respect our existing permissions when it searches internal content?
- How do you measure and report accuracy for our use case, and what do you do to reduce hallucinations?
- How is usage priced, what limits apply, and how will cost change as usage grows?
- How will you notify us of model changes, and can we test before they reach our users?
We help buyers compare AI platforms and terms in our artificial intelligence overview.
How it differs from generative AI
Generative AI is the broader category: AI that creates new content, including text, images, audio, video and code. An LLM is one kind of generative AI model, specialized in language, and is the kind behind the chat and writing features now common in business software. Image generators and many audio and video models are generative AI but not LLMs, although some newer models handle several kinds of input and output. Both sit within artificial intelligence (AI) and are built with machine learning. Natural language processing (NLP) is the older, wider field of getting software to work with human language, in which LLMs are now the leading approach.
Problems this helps with
- Every software vendor says its product now has AI built in
- Not knowing which model sits underneath an AI tool we're buying
- AI answers that sound right but turn out to be wrong
- Staff pasting company data into public AI tools
