As large language models become a central part of modern software, developers need more than a simple way to send prompts and receive responses. Production AI applications can generate thousands or millions of model calls, and each request can affect performance, cost, reliability, security, and user experience. This is where open source LLM observability becomes increasingly important.

Traditional application monitoring can show whether an application is running, but LLM applications require a deeper level of visibility. Developers need to understand which models are being used, how many tokens are consumed, how much individual requests cost, how long responses take, and what happens inside complex agent workflows. Spanlens is designed around this need, providing an open-source approach to LLM observability that can be hosted on a developer's own infrastructure.

What Is Open Source LLM Observability?

open source LLM observability refers to monitoring and analyzing the behavior of applications powered by large language models using software whose source code is openly available. Instead of treating an LLM as a black box, observability tools provide detailed information about requests, responses, tokens, latency, costs, errors, traces, and other important signals.

For an AI application in production, this visibility can make a major difference. A chatbot may suddenly become expensive because a prompt changed. An AI agent may become slower because one tool call is taking too long. A model upgrade may improve one type of response while reducing quality somewhere else. Without detailed observability, these issues can be difficult to identify.

Spanlens describes LLM observability as the practice of capturing model calls and surfacing cost, latency, token usage, and behavioral signals that determine whether an AI application remains profitable, fast, reliable, and safe.

Why LLM Applications Need Specialized Monitoring

LLM applications behave differently from traditional web applications. A conventional monitoring system may focus on HTTP requests, server performance, database queries, and infrastructure metrics. Those measurements remain useful, but they do not provide the complete picture for AI workloads.

LLM applications introduce additional variables. Token consumption directly influences costs, while different models can have significantly different pricing and response characteristics. Prompt versions can change application behavior, and AI agents can make multiple model and tool calls during a single user interaction.

This makes detailed tracing particularly valuable. With Spanlens, nested spans can represent model calls, tools, retrieval operations, and evaluation steps, allowing developers to examine an entire agent workflow rather than looking at isolated requests.

The Benefits of an Open Source Approach

One of the major advantages of choosing an open-source observability platform is control. Teams working with sensitive customer information, proprietary prompts, or regulated data may prefer to keep their AI telemetry within their own infrastructure.

Spanlens is MIT licensed and can be self-hosted using Docker. Its self-hosting offering is designed so that request logs, traces, prompts, completions, and provider credentials can remain within the organization's infrastructure.

This approach can be particularly useful for businesses that have strict data residency or security requirements. Instead of automatically sending observability information to a third-party service, organizations can operate the platform themselves and control their storage, deployment, and retention environment.

Tracking LLM Costs More Effectively

LLM costs can become difficult to manage when applications grow. Looking only at a monthly provider invoice does not tell developers which model, project, feature, customer, or request generated the highest expenses.

A modern observability platform can connect token usage with individual requests and models. Spanlens provides cost information at the model and project level, helping teams identify spending patterns and investigate unexpected increases. Its platform also highlights potential savings opportunities through model recommendations.

This type of visibility allows developers to make better decisions. For example, an application may discover that a simpler model can handle a particular task while producing acceptable results at a lower cost. Instead of guessing where money is being spent, teams can use actual production data to optimize their AI architecture.

Understanding Latency and Performance

Performance is another important part of LLM monitoring. Users expect AI applications to respond quickly, but model calls can introduce unpredictable delays.

With detailed observability, developers can examine latency across individual calls and identify slow components. This becomes especially important for AI agents because one user request may involve multiple model calls, retrieval operations, database queries, and external tools.

Spanlens presents agent workflows as nested traces, making it possible to see individual steps and identify slow portions of a workflow.

This can help engineering teams improve application performance without blindly optimizing every component.

Monitoring AI Agents and Complex Workflows

AI agents are becoming more sophisticated, and their workflows can be difficult to understand using basic request logs. An agent might first ask an LLM to create a plan, then call a database, retrieve documents, run a tool, perform another model call, and finally generate an answer.

A single final response therefore does not reveal everything that happened behind the scenes.

Open-source LLM observability platforms can provide trace-based visibility into these workflows. Spanlens supports nested spans for tools, retrieval systems, model calls, and other components, allowing developers to inspect the complete execution path.

This is valuable when debugging failed agent runs, investigating unexpected costs, or determining why a particular workflow is slower than expected.

Security and PII Visibility

AI applications can process sensitive information, making security an important part of observability. Logs themselves can become a risk if they contain personal information, credentials, or confidential prompts.

Spanlens includes PII detection that can identify information such as email addresses, credit card numbers, passport information, and other sensitive data at logging time. It also describes automatic masking of API keys found in prompts before data is stored. Teams can additionally choose not to store prompt bodies for workloads where complete request content should not be retained.

For organizations adopting open source LLM observability these controls can provide greater visibility while supporting a more deliberate approach to data handling.

OpenTelemetry Integration

Many development teams already use OpenTelemetry as part of their broader observability infrastructure. Rather than creating a completely separate monitoring ecosystem, LLM observability can be integrated into an existing tracing strategy.

Spanlens supports OTLP/HTTP ingestion and export, allowing existing OpenTelemetry tracing flows to enter the same span store while adding LLM-specific attributes to AI-related spans.

This can make adoption easier for teams that already have an established observability pipeline.

Getting Started With Spanlens

One of the notable aspects of Spanlens is its focus on simple integration. Its documentation describes a drop-in approach where developers can change the provider SDK or base URL while keeping the existing SDK interface. The platform supports integrations involving providers such as OpenAI, Anthropic, and Gemini.

For teams that prefer complete infrastructure control, Spanlens can also be self-hosted with Docker. The self-hosted platform includes capabilities such as request logging, cost tracking, agent tracing, anomaly detection, PII scanning, evaluations, and prompt experiments.

This combination of simple integration and self-hosting makes the platform suitable for developers who want observability without rebuilding their entire AI application.

Choosing the Right LLM Observability Strategy

The best observability approach depends on the application's architecture, security requirements, development stack, and scale. Some teams may prefer a hosted service for convenience, while others need self-hosting because of compliance or data-control requirements.

For developers specifically looking for open source LLM observability, licensing, self-hosting capabilities, integration options, and feature availability should all be considered. Spanlens positions itself around an MIT-licensed model, allowing teams to run the platform on their own infrastructure without relying exclusively on a hosted environment.

Conclusion

As AI applications move from prototypes into production, observability is becoming an essential part of responsible LLM development. Developers need to know what their models are doing, how much requests cost, where latency originates, how agents execute their workflows, and whether sensitive information is appearing in logs.

Open source LLM observability provides an attractive option for teams that want this visibility while maintaining control over their technology and data. Spanlens brings together request monitoring, cost tracking, latency analysis, agent tracing, evaluations, anomaly detection, security scanning, and self-hosting in an MIT-licensed platform.

For organizations building serious AI products, observability is no longer simply a debugging convenience. It is part of understanding, improving, securing, and scaling the entire LLM application. With an open-source platform such as Spanlens, developers can gain that visibility while retaining greater control over how their AI data and infrastructure are managed.