Scaler NPS: Hidden Concern Analysis

Overview

Scaler runs monthly NPS surveys across its programs. The numeric score alone is a lagging, low resolution signal. This system analyzes every non promoter's free text remarks offline with an LLM, decomposes each into a root hidden concern plus structured issue categories, and visualizes it all in a single page dashboard with seven analytical tabs.

The Problem

The real reasons learners are unhappy live in unstructured free text remarks that nobody had time to read at scale. A detractor citing two unrelated issues in one sentence was being tracked as a single, vague score.

Key Features

• Every non promoter's remarks decomposed into a root hidden concern plus up to two structured issue categories from a fixed taxonomy.
• Seven analytical tabs: Dashboard, NPS Summary, Detailed Records, MoM Analysis, Transition Analysis, Major Patterns, Visual Trends.
• Month over month, learner level movement tracking, such as Promoter to Detractor, with over indexing and lift analysis.
• Cost gated admin tooling to re run analysis on demand, with a preview, approve cost, then run contract before anything is billed.
• Pattern alerts pushed to Slack for recurring, over indexed concerns.

How It Works

1. 01Learners submit NPS survey rows into a base Google Sheet, one row per fill.
2. 02A GitHub Actions pipeline sends only new or changed non promoter remarks to the OpenAI API for analysis, never at page load.
3. 03The pipeline decomposes each remark into a root hidden concern plus up to two structured issue categories from a fixed 12 category taxonomy, then writes enriched rows to an Audit_Dump sheet.
4. 04Audit_Dump is published to the web as CSV, and the dashboard fetches and parses it client side across seven analytical tabs.
5. 05Supabase Postgres and Google OAuth gate role based access to sensitive tabs like Billing and Data Updation.

Product Thinking

• Never spend money to look at a chart. The website never calls the LLM at runtime. All AI analysis happens ahead of time in a pipeline, so the dashboard stays fast, deterministic, and safe to open to a wide audience.
• Show the why, not just the what. The product's edge is decomposed root concerns and learner level drilldowns, not another NPS number.
• Raised org wide NPS from 51 to 55 and cut weekly analysis time from 12 hours to 30 minutes by replacing manual reading of 17K+ responses with structured, drillable concerns.
• Honest empty states. When a computation cannot be made, such as no six month baseline, the UI says so instead of drawing a misleading line.

Tech Stack