DescribX Blog
How DET Scoring Works: Why Understanding Duolingo's "AI-First" Approach Matters for Test Takers image

How DET Scoring Works: Why Understanding Duolingo's "AI-First" Approach Matters for Test Takers

Disclaimer: I am not affiliated with, nor do I work for, Duolingo, Inc. The following points are not official guidelines from Duolingo, Inc. This information is based on research conducted on official Duolingo English Test blog and academic publications.

Summary

Many Duolingo English Test (DET) takers have no idea how their responses are scored. If there is one thing future test takers should understand, it's this: Duolingo is an AI-first company. This philosophy is reflected not only in how Duolingo generates course content but also in how the DET creates test materials and, most importantly, scores your responses.

This is especially critical for speaking tasks, where AI models are heavily used to evaluate pronunciation, fluency, grammar, vocabulary, coherence, and task completion .

Part 1: AI in Test Content Generation

Since 2021, Duolingo has used GPT-3 (and later GPT-4) to automatically generate test content — including reading passages, fill-in-the-blank questions, and even conversation scripts for interactive tasks .

This "human-in-the-loop" process means AI generates the content, but human experts review it for fairness, accuracy, and bias . This allows the DET to have:

  • A wider variety of test content
  • Lower costs (the DET costs about $65, compared to $200+ for many other English tests)
  • Faster refresh cycles for test questions

Part 2: AI in Speaking Task Design

The DET's speaking section includes four task types:

  1. Read, Then Speak: Read a topic, then speak your response, 20 sec prep, 30-90 sec speak
  2. Interactive Speaking: Simulated conversation with a virtual character (6-8 turns), 35 seconds per response
  3. Speak About the Photo: Describe what you see in an photo, 20 sec prep, 30-90 sec speak
  4. Speaking Sample: Extended response on a given topic, 30 sec prep, 1-3 min speak

The Interactive Speaking task is particularly innovative. Launched in 2025, it uses generative AI to simulate a real conversation — the virtual character asks follow-up questions based on what you just said .

This is different from traditional tests like IELTS or TOEFL, where you speak to a recording or a human interviewer. With the DET, AI listens to your words in real time and adapts the conversation accordingly .

Part 3: How AI Scores Your Speaking

Here is the most important part for test takers to understand.

When you respond to a speaking task, your recording goes through an automated scoring system that evaluates multiple dimensions of your speech :

  • Pronunciation: Intelligibility (can you be understood?), not native-like accent.
  • Vocabulary and Grammar: Range, sophistication, appropriateness.
  • Accuracy: Correct use of tense, articles, and word choice.
  • Coherence and Cohesion: Logical flow and organization of ideas.
  • Task Fulfillment: Did you answer the question fully?

Importantly, pronunciation is scored based on intelligibility, not imitation of a native speaker. This means you can have an accent and still score well — as long as the AI can understand what you are saying .

Part 4: Why This Matters for Test Takers

4.1 Speaking is Not About "Native-like" Perfection

Many test takers worry about sounding like a native English speaker. The DET's AI scoring model is designed to prioritize communication success (can you be understood?) over accent reduction .

4.2 The "Speech-to-Text" Step Matters

In speaking tasks, the AI first converts your speech to text. If your pronunciation is not accurate enough, the transcript will contain errors (e.g., "descripcion" instead of "description"). This will then affect how your content is scored — because the AI thinks you made grammar mistakes or used wrong words.

4.3 Interactive Speaking Tests Your Real-Time Skills

With Interactive Speaking, you cannot memorize a script. The AI asks follow-up questions based on what you just said, just like a real conversation. This tests your ability to:

  • Think on your feet
  • Respond naturally
  • Keep the conversation going

Part 5: The Research Behind the Scoring

According to research presented at NeurIPS 2024 (a top AI conference), the DET uses a sophisticated system for scoring :

  1. Items are constructed using generative AI (GPT models), then reviewed by humans
  2. Machine learning models automatically grade responses by predicting expert human ratings
  3. Items are calibrated using explanatory item response theory (IRT) models
  4. The test adapts to your ability level using Thompson sampling within a contextual bandit framework
  5. AI signals (like eye-gaze tracking) help detect cheating

This is not simple automation — it is cutting-edge AI research applied to language testing.

Conclusion: The One Thing Every DET Taker Should Know

Duolingo is not using AI as a gimmick. The entire DET — from content generation to real-time conversation simulation to automated scoring — is built on AI-first principles.

If you are preparing for the DET:

  • Focus on intelligible pronunciation, not perfect accent
  • Practice speaking on your feet (simulate conversations, not just monologues)
  • Understand that your transcript quality affects your content score
  • Read the official DET blog for updates: https://blog.englishtest.duolingo.com/

For curious and interested readers, James Sharpnack gave a keynote speech, titled "Building a High Stakes Online Test with Human-in-the-loop AI", at The First Workshop on Large Foundation Models for Educational Assessment of NeurIPS 2024. You can watch the full video here (highly recommended): https://neurips.cc/virtual/2024/107597