A large language model is a model trained on huge amounts of text so it can understand and generate language. When you enter a prompt, the AI generates text. That straightforward loop is now behind support bots, search tools, document summarizers, and in-house assistants.
LLM integration with .NET Core has become a common request because so many businesses already run their core systems on ASP.NET Core. Rebuilding those systems in another stack just to add AI makes little sense. The better path is to connect an LLM to the application you already have, using the same C# code, dependency injection, and deployment pipeline your team knows.
This guide explains how to do that. We’ll assess the key alternatives, construct a function ASP.NET Core endpoint incrementally, and address the practical challenges (safety, price, latency, hallucinations) that usually appear once the demo is running.
Why Integrate Large Language Models into .NET Applications?
Most teams do not add an LLM because it is trendy. They add it because a specific task is slow, repetitive, or hard to do with rules alone.
Automation. Classifying incoming emails, extracting fields from invoices, or drafting replies to routine tickets are tasks where an LLM can remove hours of manual work.
Customer support. An LLM can answer common questions from your own documentation and hand off to a human when it is unsure.
Intelligent search. Keyword search fails when users do not know the right keywords. LLM-backed semantic search understands what the person meant.
Content generation. Report summaries, product descriptions, and release notes can be drafted automatically and reviewed by a person.
Enterprise productivity. Virtual assistants that answer IT, HR, or policy-related questions keep workers from rummaging through company drives.
If your company is planning something larger than a single feature, a team offering AI Integration Services can help decide where an LLM actually fits and where it does not.
Popular LLM Integration Options for .NET Developers
There is no single “right” way to connect to an LLM from C#. The four options below cover most real projects, and they are not mutually exclusive.
OpenAI API
The official OpenAI .NET library (the OpenAI NuGet package) lets you call chat, embeddings, and other endpoints directly. It gives you the most control and the fastest access to new model features. The trade-off is that you are tied to one provider’s types and conventions.
Choose it when you want to move fast, you are using OpenAI models only, and you do not need an abstraction layer.
Azure OpenAI Service
Azure OpenAI gives you OpenAI models hosted inside your Azure subscription. You get enterprise features such as private networking, role-based access control, regional data residency options, and Microsoft’s compliance coverage. In .NET you use the Azure.AI.OpenAI package together with Azure.Identity.
Choose it when your organization is already on Azure, or when you need security and compliance approval for where your data lands.
Microsoft.Extensions.AI
Microsoft.Extensions.AI is a set of common abstractions, mainly IChatClient and IEmbeddingGenerator. Think of it like ILogger for AI: your application code depends on the interface, and you plug in OpenAI, Azure OpenAI, Ollama, or another provider underneath. It also supports middleware, so you can add logging, caching, and function calling around any client.
Choose it when you want provider-neutral code that is easy to test and swap later. It is the approach we use in the examples below.
Microsoft Semantic Kernel
Semantic Kernel is Microsoft’s higher-level SDK for building AI applications. It adds plugins, planning, memory connectors, and agent features on top of the model call. It is a good fit when your app needs to call several tools, chain steps, or build agents, rather than just send a prompt and read a reply.
A good rule of thumb: use the provider SDK for simple calls, Microsoft.Extensions.AI for Clean Architecture, and Semantic Kernel for orchestration-heavy features. Many organizations use LLM Development Services to build production-grade products instead of building everything in-house.
Prerequisites for LLM Integration with .NET Core
Before writing code, make sure you have:
- The .NET SDK installed (.NET 8 or later; the examples work on the current LTS release)
- Comfortable knowledge of C# and basic ASP.NET Core minimal APIs
- An API key from OpenAI, or an Azure OpenAI resource with a deployed model
- Access to NuGet to install packages
- A basic understanding of REST APIs and JSON
That is enough to follow along. You do not need machine learning experience.
Step-by-Step Guide to Integrating an LLM with .NET Core
We will build a small API that accepts a question and returns an answer from an OpenAI model.
Step 1: Create an ASP.NET Core Project
Create a minimal API project from the command line:
| dotnet new web -n LlmDemo cd LlmDemo |
A minimal API keeps the example short. The same ideas apply to controllers.
Step 2: Install the Required NuGet Packages
| dotnet add package Microsoft.Extensions.AI.OpenAI |
This package brings in the OpenAI client library and the bridge that turns an OpenAI chat client into an IChatClient. If the package is still marked as prerelease when you read this, add the –prerelease flag. Always check the package page for the current version.
Step 3: Configure API Credentials Securely
Never put an API key in source code or commit it to Git. During development, use .NET user secrets:
| dotnet user-secrets init dotnet user-secrets set “OpenAI:ApiKey” “your-key-here” |
In production, load the key from environment variables or a secret store such as Azure Key Vault. ASP.NET Core’s configuration system reads all of these through the same builder.Configuration object, so your code does not change.
Step 4: Connect to an LLM API
Register an IChatClient in Program.cs. Dependency injection then hands it to any endpoint or service that needs it.
| using Microsoft.Extensions.AI; using OpenAI; var builder = WebApplication.CreateBuilder(args); var apiKey = builder.Configuration[“OpenAI:ApiKey”] ?? throw new InvalidOperationException(“OpenAI:ApiKey is not configured.”); var model = builder.Configuration[“OpenAI:Model”] ?? “gpt-4o-mini”; builder.Services.AddSingleton<IChatClient>(_ => new OpenAIClient(apiKey) .GetChatClient(model) .AsIChatClient()); var app = builder.Build(); |
The model name comes from configuration, so you can change models without redeploying code. Failing fast when the key is missing saves you from confusing errors later.
If you use Azure OpenAI instead, only this registration changes. You create an AzureOpenAIClient with your endpoint and a credential such as DefaultAzureCredential, call GetChatClient with your deployment name, and then call As IChatClient() the same way. Using managed identity here means no API key at all, which is the better option when it is available.
Step 5: Send Prompts and Receive Responses
A chat request is a list of messages. The system message sets behavior, and the user message carries the question.
| var messages = new List<ChatMessage> { new(ChatRole.System, “You are a support assistant for a .NET software company. ” + “Answer briefly. If you are not sure, say so.”), new(ChatRole.User, “How do I enable HTTPS redirection in ASP.NET Core?”) }; var options = new ChatOptions { Temperature = 0.2f, MaxOutputTokens = 400 }; ChatResponse response = await chatClient.GetResponseAsync(messages, options); Console.WriteLine(response.Text); |
This is basic prompt engineering in .NET. A low temperature keeps answers consistent, and MaxOutputTokens caps cost and response length. A clear system prompt does more for quality than most people expect.
Step 6: Build an API Endpoint for LLM Responses
Now wrap the call in an endpoint your frontend or other services can use:
| app.MapPost(“/api/chat”, async ( ChatRequest request, IChatClient chatClient, CancellationToken cancellationToken) => { if (string.IsNullOrWhiteSpace(request.Message) || request.Message.Length > 2000) return Results.BadRequest(“Message is required and must be under 2000 characters.”); var messages = new List<ChatMessage> { new(ChatRole.System, “You are a helpful assistant. Keep answers concise.”), new(ChatRole.User, request.Message) }; var response = await chatClient.GetResponseAsync( messages, new ChatOptions { Temperature = 0.2f, MaxOutputTokens = 400 }, cancellationToken); return Results.Ok(new { reply = response.Text }); }); app.Run(); public record ChatRequest(string Message); |
Notice the input length check and the CancellationToken. The first stops people from sending huge prompts that burn tokens. The second cancels the upstream call if the client disconnects. For a chat-style interface, you can also use GetStreamingResponseAsync to send tokens to the browser as they arrive, which makes the app feel much faster.
Step 7: Test the Integration
Run the app with dotnet run, then send a request:
| curl -X POST http://localhost:5000/api/chat \ -H “Content-Type: application/json” \ -d ‘{“message”: “Explain dependency injection in one paragraph.”}’ |
Check the port printed in your console, since it varies. In automated tests, mock IChatClient. That is one of the biggest practical benefits of building on the abstraction: your unit tests never call a paid API.
Real-World Use Cases of LLM Integration in .NET Core
AI-powered chatbots. A customer-facing assistant on an ASP.NET Core site that answers product questions and creates support tickets. AI chatbot development using ASP.NET Core is usually the first project teams try.
Document processing. Upload a contract or invoice document, extract data in JSON format, and store it in SQL Server. LLMs work better on messy layouts than template-based parsers.
Intelligent customer support. Compose messages for agents, summarize long discussions, and route tickets by topic and urgency.
Enterprise knowledge assistants. The employees ask their questions in plain English, and they receive replies based on their organization’s wikis, manuals, and policies through RAG.
Automated content generation. Email drafts, product descriptions, and report summaries, always with a human review step before publishing.
Business workflow automation. An LLM reads an incoming request, decides what it is, and triggers the right workflow. When the model must take actions across several systems, it starts to look like an agent. That is where AI Agent Development Services become relevant.
If you are turning one of these into a real product rather than an internal tool, it also helps to plan the full lifecycle with AI Product Development Services.
Best Practices for Integrating LLMs into .NET Applications
Secure API key management. Use user secrets locally and Key Vault or managed identity in production. Rotate keys, and never send them to the browser. All LLM calls should go through your backend.
Error handling and retries. LLM APIs return transient errors and 429 rate-limit responses. Add retry with exponential backoff using Microsoft.Extensions.Http.Resilience or Polly. Set sensible timeouts so a slow call does not hold your request threads.
Rate limiting. Protect your own endpoint with ASP.NET Core’s built-in rate limiting middleware. Without it, one noisy user can drain your budget in minutes.
Token usage optimization. Keep prompts short, trim old conversation history, and set MaxOutputTokens. Use a smaller model for simple tasks and reserve larger models for hard ones.
Logging and Monitoring. Measure log latencies, token numbers, and error rates. Middleware for logging and OpenTelemetry is available in Microsoft.Extensions.AI. Do not log whole prompts when personal data is included in them.
Data Privacy. Determine the data that should be transferred out of your environment. Sanitize or mask any personal data prior to transfer and read the data retention policy of your provider.
Response Validation. Do not treat model outputs as gospel truth. When the output is expected to be JSON, validate it against the schema. Also, add some verification step when an output leads to an action.
Performance optimization. Stream responses for chat UIs, cache answers to repeated questions, and run independent LLM calls in parallel.
Common Challenges and How to Overcome Them
API latency. LLM calls can take several seconds. Stream the response, show a loading state, and move long jobs to background workers.
Hallucinations. Sometimes models confidently say something wrong. Base the answer on your own data with RAG, ask the model to say “I don’t know” in the absence of context, and show the sources to the user.
Complexity of integration. Making the actual API request is easy. Everything that goes around it (authentication, retries, evaluation, monitoring) is difficult. Start with one specific use case and scale from there.
Security issues. Apart from API keys, look out for prompt injection, when input from the user or retrieved documents attempts to interfere with your instructions. Treat the model output as potentially harmful input, restrict the functions the model is allowed to call, and use the same access policy you would for the user.
Scaling AI applications. Traffic surges lead to provider rate limits. Queueing, caching, and provisioning on Azure can help in addition. When scaling to a larger system, good Enterprise Software Development practices become equally important as the AI part.
Cost management. Token count increases costs, not request count. Measure token usage per feature, define budget and alerts, and audit the prompts that are unnecessarily long.
If you lack in-house capacity for this, you can Hire .NET Developers who have already shipped LLM features into production.
What’s Next for LLM Integration in .NET?
The .NET AI ecosystem is moving toward common building blocks. Microsoft.Extensions.AI is becoming the shared layer that libraries and frameworks build on, which means less provider lock-in and more reusable middleware. Semantic Kernel and Microsoft’s agent tooling are pushing toward systems that plan and use tools, not just answer questions.
In the world of business, the change moves from experimentation to production. People are challenging themselves more regarding evaluation, cost control, and governance. Local and privately hosted smaller models are becoming more powerful, which is helpful when data can’t be taken out of the business.
From the get-go, you need to depend on abstracts, keep prompts and configurations separate from code, and begin measuring for quality immediately. That is something that will last far longer than the models themselves.
Conclusion
The integration of LLMs with .NET Core is simpler than one might expect. By leveraging just a couple of packages, secure configuration, and proper initialization of the IChatClient interface, one will be able to set up an AI endpoint within a single day. However, the real challenge and benefit lies in the devil of the details.
Start small. Choose one process, create it from start to finish, measure the outcome, and then scale up. If you are looking for professional help in planning or implementing it, consider the role of AI integration in your current .NET ecosystem.
