Control Profiler

Enterprise control libraries are typically large, inconsistent, and difficult to reuse across frameworks. Controls written by different teams over many years often lack the structured attributes needed for systematic mapping, testing, and risk analysis. As a result, basic questions — Which access controls are detective? Which operate continuously? Which controls satisfy this framework requirement? — often require manual review of hundreds of records. The problem multiplies across frameworks: the same control library is repeatedly interpreted and mapped from scratch. Yet these libraries describe an institution's defenses and cannot simply be uploaded to public AI services.

I designed and built the end-to-end classification methodology and pipeline. This included a formal taxonomy covering 18 action verbs, 7 classification dimensions, and SCF-aligned domains; qualification of 14+ local AI models; a multi-pass stability methodology; human-review workflows; a design-adequacy gate; and a taxonomy feedback loop that converts downstream assessment failures into methodology improvements.

Three architectural decisions shaped the system: Local by design. Control libraries can reveal an institution's defensive posture. Classification therefore runs locally through Ollama, keeping sensitive control data inside the environment. Stability over confidence theater. Each control is classified independently three times. Agreement provides an observable quality signal; disagreement routes the decision to human review. Model-agnostic methodology. The taxonomy defines what must be classified, independent of the model performing the work. Models can change without redesigning the governance methodology.

A production classification system that profiled 569 institutional controls across all 7 dimensions in three independent runs. Every classification decision is traceable — the report links each dimension assignment back to the model's reasoning, showing which signals in the control description drove the classification. Every decision point supports human review: analysts can concur, override, or flag any dimension for any control, creating a complete audit trail of professional judgment over AI-generated classifications. Supporting artifacts include the formal specification document (v1.1), model qualification benchmarks, a taxonomy governance guide, and an interactive classification review dashboard with search, filtering, and distribution charts. The human review queue holds 76 controls where runs disagreed, each with dimension-level disagreement detail for efficient analyst resolution.

569 controls profiled  ·  7 classification dimensions  ·  86.6% full multi-dimensional agreement  ·  96.5–100% per-dimension agreement  ·  76 controls routed for human review  ·  14+ local models evaluated Every classification remains traceable to the evidence signals that produced it, and every AI-generated decision can be concurred with, overridden, or flagged by a human reviewer.

One classification pass created a reusable governance layer over 569 controls. The first downstream application — CRI v2.2 coverage assessment — is already operational. The same structured metadata can support mapping to NIST CSF, SOC 2, PCI-DSS, ISO 27001, CMMC, and additional frameworks without reclassifying the underlying controls. The control library becomes reusable governance infrastructure rather than a static collection of documents.

Why complete isolation matters — control libraries describe an organization's exact defenses, and no responsible institution should send them to a cloud AI. How a rich taxonomy and three-shot prompting make small local models (9B–32B) perform classification that would normally require frontier models. Why "stability over confidence theater" — three independent runs revealing disagreement — provides an objective quality signal that single-pass classification cannot. The taxonomy feedback loop — how CRI assessment failures traced back to missing action verbs, which drove vocabulary expansion. Why one classification pass enabling unlimited framework mappings changes the economics of compliance.