A reactive evaluation system that transforms raw content into auditable ethical evaluations through a deterministic, transparent, and scalable pipeline.
Watch how content flows through the EthiCompass system in real-time
Frontend stores conversation datasets to S3 via LakeFS
API call to /run/{metric_id} initiates evaluation
Normalized dataset object with metadata created
Async processing and job tracking
7 dimensions analyze independently
Scorecard, decisions, and explanations
Immutable log for compliance
Frontend stores conversation datasets to S3 via LakeFS. Each dataset contains session_id, assistant_id, language, context, and conversation Q&A pairs. LakeFS provides version control and returns commit_ids.
Triggers detect conditions and create Samples from raw content
Submit datasets via LakeFS with commit_ids
Periodic scans of configured content sources
External systems notify via webhook
User-initiated through interface or API
Content detection in real-time
Normalized content objects ready for ethical evaluation
sample_idstringUnique identifier (UUID)
job_idstringJob identifier for tracking
datasetarrayConversation sessions array
sourcestringTrigger source type
source_metadataobjectTrigger and callback info
evaluation_metadataobjectPriority and policy versions
statusenumpending | evaluating | completed
Each metric independently analyzes samples via cloud functions
Fairness and bias detection
Harmful language identification
Clarity and transparency
PII and data protection
Accuracy and verification
Reliability and consistency
Compliance with regulations
Clear decisions with full transparency and audit trails
Content passes all thresholds
Minor issues, proceed with caution
Requires human review
Critical issues detected
Scores for each of the 7 dimensions
Detailed reasoning for each flag
Actionable steps for improvement
Experience deterministic, transparent, and compliant AI evaluation