Paulo Dichone

Datadog LLM Observability: Monitor & Trace AI in Production

Build in-demand skills with a full year of Coursera Plus, now $239. Save now.

Paulo Dichone

Datadog LLM Observability: Monitor & Trace AI in Production

Paulo Dichone

Instructor: Paulo Dichone

Included with Coursera PlusLearn more

Gain insight into a topic and learn the fundamentals.
Intermediate level

Recommended experience

9 hours to complete
Flexible schedule
Learn at your own pace
Gain insight into a topic and learn the fundamentals.
Intermediate level

Recommended experience

9 hours to complete
Flexible schedule
Learn at your own pace

What you'll learn

  • Instrument LLM applications with Datadog's ddtrace SDK for complete visibility into prompts, completions, and token usage.

  • Trace complex AI agent workflows, including multi-turn conversations and tool calls, with enterprise-grade debugging.

  • Implement production evaluations, custom LLM-as-a-judge tests, and configure cost tracking dashboards.

Details to know

Shareable certificate

Add to your LinkedIn profile

Recently updated!

November 2026

Assessments

11 assignments

Taught in English

See how employees at top companies are mastering in-demand skills

 logos of Petrobras, TATA, Danone, Capgemini, P&G and L'Oreal

There are 9 modules in this course

Understand the core concepts of LLM observability and why it is critical for enterprise environments. You will explore Datadog's core capabilities and interface, laying the foundation for tracing and monitoring AI applications in production.

What's included

3 videos2 assignments

Configure your local environment and initialize the Datadog SDK to start tracing LLM requests. You will learn about different span types and auto-instrumentation capabilities to establish a connection with the Datadog platform.

What's included

4 videos2 assignments

Enrich traces by creating custom LLM spans with annotations, tagging, and metadata. You will also learn how to instrument multi-step workflows and retrieve document segments within RAG pipelines for comprehensive tracing.

What's included

5 videos2 assignments

Gain deep visibility into multi-agent systems and agentic decision-making paths. You will practice tracing complex agent interactions, tool usage, and learn debugging strategies to resolve agent-specific issues in production.

What's included

3 videos1 assignment

Develop evaluation workflows to continuously assess the performance, accuracy, and safety of your LLMs. You will run experiments, create golden datasets, and set up quality monitoring dashboards to compare prompt performance.

What's included

7 videos1 assignment

Track, analyze, and optimize LLM cost metrics within Datadog. You will learn to use automated dashboards to observe token consumption, monitor API usage, and implement cost-saving architectural strategies.

What's included

2 videos1 assignment

Incorporate robust security protocols into your LLM pipelines, including PII redaction and sensitive data scanning. You will evaluate production deployment architectures and checklists to ensure safe enterprise delivery.

What's included

6 videos1 assignment

Summarize your achievements throughout the course and plan your next steps for continuing your education in LLM observability and production monitoring.

What's included

1 video1 reading

Assess your ability to instrument LLM applications, interpret Datadog traces and evaluation results, and apply cost optimization, security, and production monitoring practices.

What's included

1 assignment

Instructor

Paulo Dichone
Paulo Dichone
1 Course4 learners

Offered by

Paulo Dichone

Why people choose Coursera for their career

Felipe M.

Learner since 2018
"To be able to take courses at my own pace and rhythm has been an amazing experience. I can learn whenever it fits my schedule and mood."

Jennifer J.

Learner since 2020
"I directly applied the concepts and skills I learned from my courses to an exciting new project at work."

Larry W.

Learner since 2021
"When I need courses on topics that my university doesn't offer, Coursera is one of the best places to go."

Chaitanya A.

"Learning isn't just about being better at your job: it's so much more than that. Coursera allows me to learn without limits."

What brings you to Coursera today?

Frequently asked questions