AI tools for “observability”
93 results (tools & skills) · 331 ms · understood as by meaning
LangSmith Observability gives complete visibility into agent behavior with tracing, real-time monitoring, alerting, and high-level insights into usage.
A concise feature matrix for deciding which PlanetScale safety, observability, and automation recommendations apply by engine.
Analyze the most expensive users in AI observability and explain why they cost so much. Use when the user asks about top spenders, expensive users, per-user LLM cost, user-level cost drivers, or patterns behind high AI…
…continuously-running online evaluations in PostHog AI observability, grounded in real failure modes you've identified. Use when the user wants evaluations that automatically score new generations or whole traces…
…distributions, trace IDs, or span attributes — not AI observability traces or product logs. Uses posthog:apm-service-health, posthog:query-apm-spans, posthog:apm-trace-get, posthog:apm-spans-sparkline,…
Investigate AI observability clusters — understand usage patterns in AI/LLM traffic, compare cluster behavior, compute cost/latency metrics, and drill into individual traces within clusters.
Add PostHog AI Observability to trace AI model usage. Use after implementing LLM features or reviewing PRs to ensure all generations are captured with token counts, latency, and costs. Also handles initial PostHog SDK…
Investigate AI observability evaluations — `hog` (deterministic code-based), `llm_judge` (LLM-prompt-based), and `sentiment` (user-message sentiment). Find existing evaluations, inspect their configuration, run them…
Set up and troubleshoot Pydantic Logfire frontend observability and session replays in JavaScript/TypeScript browser applications. Use for browser errors, RUM, Web Vitals, replay recording, privacy controls, or reducing…
…Use when the user pastes a trace or session URL (e.g. /ai-observability/traces/<id> or /ai-observability/sessions/<id>), asks to debug a trace, figure out what went wrong, check if an agent used a tool correctly,…
Add Pydantic Logfire observability to application code — traces, logs, metrics, and AI/agent spans. Use when the user asks to add or configure Logfire, observability, tracing, logging, or monitoring; maximize useful…
…instrumentation, infrastructure, evals, frontend observability, Gateway, or coding agents. Use for "set up Logfire", "add Logfire to my project", "get me set up properly with Logfire", "send as much data as…
Diagnoses Qdrant production issues using metrics and observability tools. Use when someone reports 'optimizer stuck', 'indexing too slow', 'memory too high', 'OOM crash', 'queries are slow', 'latency spike', or 'search…
Guides Qdrant monitoring and observability setup. Use when someone asks 'how to monitor Qdrant', 'what metrics to track', 'is Qdrant healthy', 'optimizer stuck', 'why is memory growing', 'requests are slow', 'set up…
Signals scout for PostHog AI observability. Watches LLM traces for cost, latency, error, volume, and eval-performance regressions.
Signals scout for observability gaps — significant event volumes with no insight, dashboard, or alert coverage. Recommends new insights, dashboards, or alerts as the product evolves.
…`authoring-scouts` for edits, `exploring-scouts` for run observability, and `inbox-exploration` for report triage. Trigger on "work with my scouts", "get more out of scouts", "have a scout watch X", "tell me if Y…
…"investigate incident", or any request to pull observability or monitoring data. Also triggers when the user mentions APM, distributed tracing, golden signals, Errors Inbox, or synthetic monitors. Always…
…correctness, file transfer, bulk work, rate limits, and observability; not one-off CLI or MCP content…
Sets up CloudWatch observability for the first time - Omni (CloudWatch Application Observability) and classic CloudWatch. Omni: creating a Space or Domain; access grants (who has access, at what level), access profiles…