Udemy – Evaluating & Improving Generative & Agentic AI

The AI Literacy Specialization Program is one-of-a-kind hierarchical & cognitive skills based curriculum that teaches artificial intelligence (AI) based on a scientific framework broken down into four levels of cognitive skills.

Part 3: Analyze and Evaluate combines the below two cognitive skills –

• Analyzing (examining model’s outputs to identify biases that may have been learned from the training data)
• Evaluating (assessing the performance, ethics, and overall effectiveness of an AI system)

Whether you’re a policymaker, red teamer, LLM safety auditor, risk manager, AI developer, or a certification aspirant (AIGP, RAI, AAIA), this program gives you what theory alone cannot: applied, practice-ready evaluation and mitigation skills.

The course is structured into four progressive modules: what to evaluate, how to evaluate, how to mitigate, and real-world case studies:

Introduction:

• Why Evaluate AI?
• Who Should Evaluate & When?

What to Evaluate:

• 1.1) Standard Safety Evaluation
• 1.2) Frontier Capabilities & Misuse
• 1.3) Misalignment
• 1.4) Structural & Multi-Agent Risks

How to Evaluate:

• 2.1) Overview
• 2.2) Benchmarks & Datasets
• Benchmark Repository v0.1
• 2.3) Evaluation Metrics
• 2.4) Evaluation Techniques
• 2.5) Red Teaming 101
• Evaluation Toolkit Repository v0.1

How to Mitigate:

• 3.1) How to Mitigate
• Mitigation Techniques Repository v0.1

DOWNLOAD LINK: Evaluating & Improving Generative & Agentic AI