OpenMatter-Network avatar

ai-personnel-assessment

Auditing AI/ML personnel assessments for fairness, bias, validity, and utility, using the 12-component psychological-audit framework of Landers & Behrend (2023), 'Auditing the AI Auditors.'

GitHub stars
10

Install

/plugin install ai-personnel-assessment@io-skills

Run it inside Claude Code after adding the marketplace with /plugin marketplace add OpenMatter-Network/agent-io-skills.

Contents

Discovered from source

Skills 8

AI audit meta-components (Components 10–12)

These three components are meta because they must be considered across all the other components, not as a separate stage. They ask: in what cultural and ethical context does this system operate, and is the evidence behind every claim methodologically sound?

Component 10 — Cultural context

What it is: the broader cultural setting in which the algorithm is used, and whether affected communities had a voice in it.

Questions to ask: Has the broader cultural context been considered? Have members of the community participated in the design of systems that will affect them?

Apply it (focal example): Do power differentials exist between designers, employers, and job candidates? Have cultural assumptions been made? Will development decisions made in one culture be applied to another — and if so, how has the development process been adjusted to prevent cross-cultural application challenges?

Audit emphases:

  • Name the power asymmetry: designers and employers hold power over candidates who often can't opt out, see their data, or contest a score.
  • Flag cross-cultural transfer: a model trained/validated in one cultural or linguistic context and deployed in another (different dialects, norms, nonverbal behavior) without adjustment is a high-risk finding. Connects to feature-engineering choices (dialect/NLP) in ai-model-development-audit and to linguistic/cultural equivalence in candidate-accommodations.
  • Ask whether affected communities participated in design — absence is itself a finding.

Component 11 — Respect (conformance to ethical standards)

What it is: whether the algorithm is developed and used in conformance with generally accepted ethical standards.

Questions to ask: Does its use conform to accepted ethical standards — e.g., the Standards, the SIOP Principles, the OECD Principles on AI, and the UGAI?

Apply it (focal example): What ethical standards do the developers claim to have followed? Is there evidence of decisions actually made following that framework? What evidence is there that individual fairness was a priority during development?

Audit emphases:

  • Distinguish professed standards from demonstrated adherence — require traceable decisions, not a values statement.
  • For psychologists, the APA Ethics Code binds the work (beneficence/nonmaleficence, fidelity/responsibility, integrity, respect for rights and dignity, justice/minimizing one's own bias) — see ai-fairness-lenses, Lens 2.
  • AI-specific codes (OECD, UGAI) reference fairness/reliability/validity but don't define them precisely — so "we follow UGAI" is not self-certifying; check what was actually done.

Component 12 — Research designs

What it is: the methodological quality of the studies offered to support any claim — the integrity check underneath everything.

Questions to ask: How do the research designs (sampling, experimental design, variable choices, analysis, interpretation) of any supporting studies affect the validity of the conclusions?

Apply it (focal example): For every claim that appears to rest on empirical observation, does the study design support the claim? Were all design decisions defensible from the perspective of modern methodological research? What impact might they have had on the validity of the conclusions?

Audit emphases:

  • This is where the auditor applies standard research-methods scrutiny to the developer's own validation studies: sampling adequacy, confounds, appropriate analyses, defensible interpretation.
  • It pairs with the psychometric evaluation in ai-model-outputs-audit — Component 12 asks whether the study that produced the validity/reliability evidence was itself sound.
  • An audit's own credibility also rests here: failing to articulate the standards by which the audit was conducted can make its results uninterpretable.

Pitfalls

  • Treating culture/ethics/research-integrity as an afterthought instead of cross-cutting checks.
  • Accepting a values statement as proof of ethical adherence.
  • Missing cross-cultural transfer risk for a model moved between contexts.
  • Auditing reported results without auditing the design that produced them.
  • Ignoring power differentials that prevent candidates from contesting or understanding decisions.

Checklist

  • Power differentials among designers/employers/candidates named
  • Cultural assumptions and cross-cultural transfer risks evaluated; adjustments verified
  • Community participation in design assessed
  • Claimed ethical frameworks (Standards/Principles/OECD/UGAI/APA Ethics) identified
  • Demonstrated (not just professed) adherence evidenced via traceable decisions
  • Every empirical claim's underlying study design scrutinized for methodological defensibility
  • Audit's own fairness/measurement standards articulated for interpretability

See also

ai-fairness-lenses (legal/ethical/moral lens) · ai-audit-planning · ai-model-outputs-audit (psychometric counterpart to Component 12) · ai-audit-reporting · candidate-accommodations (linguistic/cultural equivalence)

Source: Landers & Behrend (2023), Table 1 (Components 10–12, "Meta-components").

AI audit planning

Scope the psychological audit before doing it. A psychological audit is an impartially conducted conceptual and empirical evaluation of claims about psychological characteristics — traits, attitudes, emotions, behaviors, and other quantities not directly observable — as measured or predicted by algorithms. The target characteristics are set by the developers, and may not correspond to what the algorithm ultimately measures or predicts — so the audit must surface that gap.

An audit must be both conceptually grounded in psychometric theory and empirically tested, using a research design appropriate to the claims and interpreted according to modern standards of test evaluation.

Step 1 — Identify the claim(s) to evaluate

Audits can evaluate claims of any type, but are most valuable for:

  • Validity — is measurement/prediction of the targeted characteristic done consistently and accurately? (Most tractable — broad agreement exists via the psychometrics community and the Standards / SIOP Principles.)
  • Utility — does the algorithm provide value from its implementation? (Even within one discipline there's disagreement on how to conceptualize value; frameworks include multiattribute utility, ROI, litigation liability, triple-bottom-line. In medicine, value ≈ lives saved/suffering reduced.)
  • Lack of bias — the most complex, because the claim can be made through any of the three fairness lenses. Failing to articulate the precise standards of fairness/bias can render the audit uninterpretable across disciplines. Resolve this with ai-fairness-lenses first.

Write each developer claim as an explicit, testable statement (e.g., "this algorithm predicts job performance equally well for all groups using appropriate modeling techniques") — then design to evaluate it. In the focal example, that exact claim is questionable and should be audited.

Step 2 — Establish auditor stance and credibility

Auditors come in three types; none is automatically credible:

  • Internal — employees of the developing company with psychometric/algorithm expertise.
  • External — consultants hired by the developer.
  • Independent — brought in externally, often (not exclusively) by a regulator.

It's intuitive to rank these by credibility, but access to data and documentation varies greatly even within a type (NDAs can restrict how deeply anyone can probe). So evaluate every auditor — regardless of source — on: their stated measurement standards and definitions of bias, whether they applied them in due course, and the terms of access and nondisclosure (what may be withheld at the company's discretion should itself be disclosed). Claims that "an algorithmic system has been audited and is therefore credible" should be viewed skeptically.

Step 3 — Decide purpose & timing: formative vs. summative

  • Formative (during development) — a particularly valuable purpose is to guide developers so algorithms can be improved before/while deployed. Careful adherence to core auditing components at all stages of development, plus complete documentation, can diminish or even preclude the need for post-hoc auditing. Normalize routine internal/external auditing as a public good.
  • Summative (post-hoc) — verify developer claims and identify adverse effects after deployment; recommend corrective actions to remove/minimize them.

Prefer building auditing into the development lifecycle, not just bolting it on afterward.

Step 4 — Identify the audience and disclosure plan

The audit's purpose is often driven by its intended audience. Plan to produce results in multiple formats for all relevant audiences (a precise technical report and a layperson-friendly summary for those whose predictions are affected) — see ai-audit-reporting. Decide the release policy up front: in general, unless there's a compelling, transparently stated reason, audits in the public interest should be released; withholding risks the credibility of the audit, the company, and the auditors.

Step 5 — Secure access and define the evidentiary base

Confirm what data, documentation, and code you can examine, and record access limits as findings in their own right. The audit will span the 12 components across three categories (see the collection README and Table 1):

  • Models (1–6) → ai-input-data-and-design-audit, ai-model-development-audit, ai-model-outputs-audit
  • Information & perceptions (7–9) → ai-claims-and-stakeholder-audit
  • Meta (10–12) → ai-audit-meta-components

Table 1 is not exhaustive — a framework for the most important issues. When a claim can't be fully evaluated within it, ask additional questions tied to relevant professional standards.

Note on regulatory "bias audits"

Some laws mandate audits (e.g., NYC Local Law 144 requires an annual "bias audit" of automated employment decision tools; many bills leave "bias" undefined so the term stays adaptable). A compliance audit and a scientifically meaningful psychological audit are not the same — satisfying a statute's checklist does not establish validity or lack of bias under Lens 1/3. Scope both explicitly and don't let one masquerade as the other. (Coordinate with counsel on jurisdictional law.)

Pitfalls

  • Starting empirical work before defining the fairness standard (ai-fairness-lenses).
  • Treating "it was audited" as proof of fairness.
  • Ignoring NDA/access limits instead of reporting them as findings.
  • Auditing only post-hoc when formative auditing would have prevented the harm.
  • Conflating a regulatory compliance audit with a validity/bias audit.

Checklist

  • Each developer claim written as an explicit, testable statement
  • Claim type(s) classified (validity / utility / lack of bias)
  • Fairness standard defined via ai-fairness-lenses
  • Auditor type, access terms, and NDA limits documented and disclosed
  • Formative vs. summative purpose set; lifecycle integration considered
  • Audience(s) and release policy decided
  • 12-component scope mapped to downstream skills; access gaps noted
  • Regulatory vs. scientific audit scope separated

See also

ai-fairness-lenses (prerequisite) · ai-audit-reporting · all model/stakeholder/meta audit skills · validation-planning · technical-validation-report

Source: Landers & Behrend (2023), "Designing an Effective Psychological Audit"; auditor types & credibility; formative auditing; "Major Components of an Audit" / Table 1.

AI audit reporting

An audit only creates value if its results are communicated to the right audiences and, where the public interest is at stake, released. Reporting is also where the auditor's own credibility is established or lost.

Produce results in multiple formats (dual-audience)

Present results in multiple formats to meet the needs of all relevant audiences. At minimum:

  1. A precise, comprehensive technical report for review by testing professionals — detailed enough that another competent professional could evaluate and, ideally, reproduce the audit's reasoning. Mirror the structure of a technical-validation-report, but organized around the 12 components and the claims evaluated.
  2. A layperson-friendly summary for whomever the audit's predictions will directly affect (e.g., candidates) — clear, accurate, non-technical, addressing the fairness concerns in the terms those audiences actually use (justice/transparency — see ai-fairness-lenses).

One format cannot serve both audiences; write both.

Suggested technical-report structure

  • Claims evaluated — each developer claim as an explicit statement, classified as validity / utility / lack-of-bias (from ai-audit-planning).
  • Fairness standard(s) — the lens(es) and precise definitions used (ai-fairness-lenses), stated so conclusions are interpretable across disciplines.
  • Findings by component (1–12) — model (1–6), information & perceptions (7–9), meta (10–12), each with evidence, gaps, and access limitations encountered.
  • Auditor credibility statement (see below).
  • Recommended corrective actions — to remove or minimize adverse effects, and (for formative audits) to improve the model.
  • Limitations — what could not be evaluated, and why (NDA, missing documentation, small intersectional subgroups, etc.).

Establish credibility in the report

No auditor or audit is automatically credible. "This system has been audited and is therefore credible" deserves skepticism. So the report must let readers judge the audit itself by disclosing:

  • the measurement standards and definitions of bias/fairness applied, and that they were applied in due course across the components;
  • the auditor's type and relationship to the developer (internal / external / independent);
  • the terms of access and nondisclosure — including what could be withheld at the company's discretion (the existence and scope of withholding should itself be disclosed).

Access and documentation vary even within auditor type, so transparency about access is essential to interpreting the findings.

Decide on release — default to disclosure in the public interest

Unless there is a compelling, transparently stated reason not to, an audit whose results are in the public interest should be released. Organizations may choose otherwise, but doing so risks the credibility of the audit, the company that built the algorithm, and the auditors. Public-facing, transparent, open audits are especially warranted when a system has outsized societal impact.

Normalize routine auditing. Treat regular internal and external auditing as a public good that raises the probability algorithmic systems in general are valid, valuable, and fair — and that builds public trust. Formative auditing folded into development, with complete documentation, can diminish or even preclude the need for post-hoc audits.

Pitfalls

  • Shipping only a technical report (affected people can't use it) or only a summary (professionals can't evaluate it).
  • Implying "audited = trustworthy" without exposing standards and access terms.
  • Hiding what was withheld under NDA instead of disclosing that it was withheld.
  • Suppressing a public-interest audit, harming everyone's credibility.
  • Reporting findings without actionable corrective recommendations.

Checklist

  • Technical report (component- and claim-structured) produced for professionals
  • Layperson summary produced for affected audiences, in their fairness terms
  • Fairness lens(es) and precise standards stated for interpretability
  • Auditor type, relationship, and access/NDA terms disclosed (incl. what was withheld)
  • Findings, gaps, and limitations reported honestly per component
  • Corrective actions recommended (and, if formative, improvement guidance)
  • Release decision made; public-interest default to disclosure applied and any non-release justified

See also

ai-audit-planning (audience & release policy set up front) · ai-fairness-lenses · all model/stakeholder/meta audit skills · technical-validation-report (structure parallel)

Source: Landers & Behrend (2023), "Designing an Effective Psychological Audit" — multiple-format reporting, releasing results in the public interest, normalizing routine auditing, and auditor credibility.

AI claims & stakeholder audit (Components 7–9)

The model can be technically sound and still be mis-described or harmful in use. This category shifts from the model's internals to how information about it is presented, understood, and experienced by three parties. It leans heavily on the individual-attitudes (justice) lens — see ai-fairness-lenses.

Component 7 — First-party interpretation (developer claims)

What it is: the messaging the algorithm developer puts out about the model.

Questions to ask: Does all messaging from the developer logically, honestly, and transparently follow from answers developed elsewhere in the audit?

Apply it (focal example): Does the developer claim the model predicts job performance? What evidence in the audit forms the basis of that claim? Are important details left out?

Audit emphases:

  • Cross-check every public/marketing/sales claim against the model-audit findings (Components 1–6). A claim unsupported by the evidence — or that omits material caveats (range restriction, k-fold-only validation, untested intersectional subgroups) — is a finding.
  • This is informational justice in action: transparency about what is assessed and what is done with the data shapes how fair the system is perceived to be.

Component 8 — Second-party effects (those assessed)

What it is: impact on the people directly affected by the algorithm's use (candidates).

Questions to ask: Who is directly affected, and how have their outcomes and reactions been assessed? What is the relative impact of acting on false positives versus false negatives on second parties?

Apply it (focal example): How do non-selected applicants react to learning the algorithm did not score them high enough to be selected? What information is communicated to them, and how do they evaluate that information?

Audit emphases — use justice theory (Lens 1):

  • Procedural justice rules the developer may be violating: opportunity to perform, job-relatedness/ face validity (do candidates believe facial expressions predict performance?), reconsideration/ appeal (algorithmic decisions that can't be appealed), two-way communication (AI replacing human interaction), and propriety (some find AI decisions morally inappropriate).
  • Distributive justice: which rule (equality/need/equity) do affected people apply to the outcome?
  • Interactional justice: interpersonal (respect/dignity) and informational (adequate explanation, e.g., an explanatory video before data collection).
  • False positives vs. false negatives are not symmetric for second parties. A false negative (a qualified candidate wrongly screened out) harms the individual; weigh it explicitly against false positives rather than optimizing a single accuracy number. Tie this to the error-cost reasoning in selection-decisions-and-scoring.

Component 9 — Third-party understanding (outside observers)

What it is: how outside observers perceive and evaluate the system.

Questions to ask: How have perceptions and evaluation by outside observers been assessed and incorporated? Have outside regulatory groups and community organizations been consulted?

Apply it (focal example): How do experts in employment law view the documentation and performance of the algorithm? How does the public view this use of algorithms?

Audit emphases:

  • Solicit employment-law review (differential prediction, disparate impact, differential treatment — Lens 2) and community/regulatory input, rather than assuming internal sign-off suffices.
  • Public trust is part of the value proposition; transparent, outward-facing evaluation raises it (and is a stated benefit of normalizing audits).

Pitfalls

  • Auditing the model but never checking the claims made about it against the evidence.
  • Optimizing overall accuracy while ignoring the asymmetric cost of false negatives to candidates.
  • Treating candidate reactions as PR rather than justice evidence relevant to fairness.
  • No appeal/reconsideration path; no two-way communication.
  • Assuming legality without consulting employment-law experts or affected communities.

Checklist

  • Every developer claim cross-checked against Component 1–6 findings; omissions flagged
  • Informational transparency (what is measured / what is done with data) assessed
  • Directly affected parties identified; reactions/outcomes assessed
  • Procedural, distributive, and interactional justice rules evaluated
  • False-positive vs. false-negative impacts weighed for second parties
  • What is communicated to non-selected candidates reviewed (incl. appeal/reconsideration)
  • Employment-law expert and community/regulatory perspectives obtained
  • Public-perception considerations documented

See also

ai-fairness-lenses (justice; legal lens) · ai-audit-reporting (communicating to these audiences) · ai-audit-meta-components · fairness-and-bias-analysis · selection-decisions-and-scoring (error costs) · administration-documentation (candidate communications, feedback)

Source: Landers & Behrend (2023), Table 1 (Components 7–9, "Components relating to information and perceptions").

AI fairness lenses

Before you can audit an AI assessment for "fairness" or "bias," you must define which meaning you are using and defend it. Fairness and bias have *very different meanings for different audiences and disciplines. Failing to articulate the precise standard "can render the results of an audit uninterpretable across disciplinary lines." This skill gives you the three lenses to choose among and combine.

Critically: the same word splits. A critic who calls an AI "unfair" may mean what another person calls "bias"; and "bias" is itself loaded. Name the lens explicitly in any audit.

Lens 1 — Individual attitudes (justice theory)

The most-invoked lens in public discourse. Use organizational justice (a tripartite perception framework — cognitive, perceptual, emotional) to structure "this feels unfair" claims:

  • Distributive justice — perceived fairness of outcomes (who gets hired). Judged against rules of equality (same outcome to all), need (most to those who need most), or equity (outcomes proportional to inputs/contribution). People apply different rules to the same decision, informed by cultural and social values. Most public "AI is unfair" complaints are implicitly distributive.
  • Procedural justice — perceived fairness of the rules and procedures used to decide. Five rules often implicated by AI decisions (Ford et al.):
    1. Opportunity to perform — candidate feels they got a fair chance to demonstrate their value.
    2. Job-relatedness / face validity — e.g., do people believe facial expressions in a video interview relate to future performance? (And is evidence of relevance required to justify inclusion?)
    3. Reconsideration / appeal — algorithmic decisions that can't be appealed violate this.
    4. Two-way communication — violated when AI replaces face-to-face human interaction.
    5. Propriety — some simply believe AI-based decisions are morally inappropriate.
  • Interactional justice — perceived fairness of the interaction with decision-makers, split into interpersonal (treated with respect/dignity) and informational (given adequate information about the decision and how it was reached). Informational justice is directly affected by transparency about what is assessed and what is done with it; interpersonal justice by the presentation strategy (e.g., an explanatory video before data collection).

Use this lens to predict and diagnose candidate reactions — see ai-claims-and-stakeholder-audit (second-party effects).

Lens 2 — Legality, ethicality, and morality

Fairness as alignment with shared human values and established professional/legal guidelines — "governed by a sense of responsibility to others." Two streams:

  • Ethical / moral. A fair system conforms to relevant and established professional guidelines.
    • The APA Ethics Code (2017) five principles bind any psychologist working on such a system: beneficence/nonmaleficence, fidelity/responsibility, integrity, respect for people's rights and dignity (treat people equitably regardless of personal/group characteristics), and justice (address and minimize one's own biases). Here "bias" ≈ a goal of impartiality, lacking individual prejudices and cognitive biases.
    • AI-specific codes — OECD Principles on AI (2019), Universal Guidelines for AI (UGAI, 2018) — reference fair/unbiased decisions but generally don't define them precisely; UGAI names reliability, validity, and data quality and uses bias / discrimination / unfairness interchangeably. These tend to leave "bias" vague so it stays applicable as standards evolve — effectively delegating the technical definition to Lens 3.
  • Legal. Laws on discrimination in hiring/housing/admissions often have precise technical definitions built on statistical concepts and case law. For employment, legally establishing test bias generally relies on differential prediction (comparing regression lines across legally defined classes — race, sex, national origin), used to identify the source/justifiability of disparate (adverse) impact (differential selection rates). Contrast differential treatment — explicitly treating class members differently (e.g., awarding bonus points to a group, or modeling class membership as a predictor). Note: advocacy, bills, and even policy often muddle these decades-old distinct concepts, sometimes deliberately leaving "bias" vague.

Lens 3 — Technical domain-embedded meanings of "bias"

All technical definitions share a root: bias = inaccuracy in estimating a population value from sample data, where error splits into random vs. systematic. But the disciplines diverge:

  • Statistics — sampling bias. Systematic error in sample representation vs. the population. A performance estimate (e.g., R² = .5) won't generalize if the training sample differs nonrandomly from the target population (the classic undersampled-minorities / facial-recognition database problem). Bias here is a consequence of improper sampling; fix by better/representative sampling or oversampling underrepresented groups (not guaranteed to work in practice).
  • Machine learning — the bias-variance tradeoff. ML deliberately introduces bias (e.g., ridge, lasso, elastic net penalize large weights) to reduce overfitting and improve out-of-sample predictive accuracy. Here bias can be positive and desirable — a property of a well-engineered model. Prioritizing "unbiased estimates" above all (as mainstream psychology's low-bias/high-variance procedures do) can hurt out-of-sample prediction. The cost: individual coefficients are no longer cleanly interpretable.
  • Psychometrics — measurement bias / invariance. Differences in measurement characteristics across identified groups, commonly assessed as measurement invariance (CFA latent factors) or IRT item parameters. Psychometric bias may or may not be problematic: if a measure is meant to assess a construct on which groups genuinely differ (e.g., educational attainment shaped by systemic opportunity differences), group differences are expected and the test may be biased-but-fair. Note the psychometric caution that differential prediction is not a sufficient condition for bias — a test can show differential prediction without problematic measurement properties (the "six sigma"/manager age example: younger applicants score lower because they've had less exposure, yet if managerial experience is job-related the differential prediction may still be considered fair).

How to use the lenses in an audit

  1. State the claim being evaluated and who is raising the fairness concern.
  2. Pick the lens(es) that match — concerns may invoke any or all three.
  3. Define the standard precisely within that lens (which justice rule? which legal test? which technical bias?) and write it into the audit so conclusions are interpretable.
  4. Don't equivocate — a finding of "no measurement bias" (Lens 3) does not answer a distributive or procedural-justice complaint (Lens 1) or a legal disparate-impact question (Lens 2).
  5. Carry the chosen standard into the model, stakeholder, and meta audits.

Pitfalls

  • Auditing "fairness" without naming a lens → uninterpretable results.
  • Treating any group difference as "bias" (psychometrically it may be expected and fair).
  • Assuming "bias is always bad" — in ML, intentional bias improves generalization.
  • Conflating differential prediction, disparate impact, and differential treatment.
  • Answering a procedural/distributive-justice complaint with a purely statistical result.

Checklist

  • Fairness concern's source/audience identified
  • Lens(es) selected and named in the audit
  • Precise standard defined within each lens (justice rule / legal test / technical bias type)
  • Cross-lens gaps acknowledged (statistical result ≠ justice answer)
  • Standard documented and propagated to downstream audit components

See also

ai-audit-planning · ai-model-outputs-audit (subgroup differences, measurement bias) · ai-claims-and-stakeholder-audit (justice/candidate reactions) · fairness-and-bias-analysis (predictive vs. measurement bias in the Principles)

Source: Landers & Behrend (2023), "Defining Fairness and Bias"; Lenses 1–3; "Contrasting Statistics, Machine Learning, and Psychometrics Perspectives."

AI input-data & model-design audit (Components 1–2)

The two far-reaching, hard-to-reverse decisions at the base of any AI assessment. Get these wrong and everything built on top inherits the flaw — yet they're often the least documented. Define your fairness standard first (ai-fairness-lenses) and scope via ai-audit-planning.

Component 1 — Input data

What it is: the validity and generalizability of the dataset(s) used to train the model, relative to the population where predictions will actually be generated and used.

Questions to ask: How were input data collected, in terms of population and research design? How did those factors affect data quality? Evaluate the population the data were drawn from, the sampling techniques, and the adequacy of the data for the analyses later used to support claims.

Apply it (focal example — algorithmic video interview): Were input data collected from job incumbents? How do incumbents differ from applicants? Is there range restriction on variables of interest? A model trained on a convenient source of existing candidates/incumbents is no guarantee of representativeness for future applicants — so a key audit goal is to evaluate how the developers investigated that assumption and concluded it was not a problem.

Audit emphases:

  • This is the AI face of classic validation concerns: population selection and the concurrent-vs-predictive design choice directly shape what can be inferred (see SIOP Principles via criterion-related-validation).
  • Sampling bias (Lens 3, statistics): nonrandom under-/over-representation (e.g., undersampled minorities) makes the model weight patterns of the most-represented group; "we'll just collect a more diverse database / oversample" is not guaranteed to fix it.
  • Every training dataset counts — including later subset datasets used for more specific modeling challenges. Each must meet the same quality standard, embedded in a meaningful data engineering framework. Don't let a well-vetted primary dataset launder a sloppy secondary one.

Component 2 — Model design

What it is: the initial modeling decisions that are difficult to test or change later. The three most consequential:

  1. the nature of the criterion modeled,
  2. the set of predictors/features provided to the model,
  3. the initial set of algorithms used.

Even when the data are adequate (Component 1), the criterion and the available predictor set drive many subsequent steps. These choices may be made conceptually, empirically, or heuristically — ask which, and demand the rationale.

Questions to ask: What drove the initial model choices (criterion, predictor set, algorithm)? Were they informed by theory or empirically derived? If empirically derived, from what data — and of what quality — were they derived?

Apply it (focal example):

  • Criterion / "ground truth" validity is the crux. "Performance data were available" is not enough — audit the construct validity of that performance measure as a valid representation of performance, including a clear, unambiguous, scientifically defensible definition of what performance is. In computer science this is framed as choosing a "ground truth," often without any explicit consideration of construct validity — name and close that gap.
  • Predictor inclusion needs a rationale. Why are word choice, answer content, voice patterns, and visual data included as predictors? Are those decisions theoretically supported? Even though weak features can be dropped in later analytic stages, their initial inclusion is a design-stage decision that creates technical debt and sunk-cost pressure — and, in the focal example, drags in construct-irrelevant correlates (wrinkles→age, skin tone→race/ethnicity, makeup→gender) that the designer may not even realize the model is exploiting.

Audit emphases:

  • Tie criterion definition back to a real analysis of work (work-analysis) — a defensible ground truth requires it.
  • Watch criterion reconceptualization / weighting decisions: down-weighting task vs. up-weighting contextual performance can reduce adverse impact, but such weighting has validity implications (Sackett et al., 2008) and must be carefully articulated and defended, not silently chosen.
  • Flag predictors that proxy protected characteristics; inclusion-by-default is a finding.

Pitfalls

  • Accepting "data were available" without auditing the criterion's construct validity.
  • Treating incumbent-trained models as valid for applicants without addressing range restriction.
  • Letting a clean primary dataset excuse unvetted subset datasets.
  • Reading "features can be pruned later" as a reason not to justify their initial inclusion.
  • Missing predictor proxies for protected class because the developer "didn't intend" them.

Checklist

  • Training population vs. deployment population compared; representativeness investigated
  • Sampling technique evaluated for sampling bias; range restriction assessed
  • Concurrent-vs-predictive design implications stated
  • All training datasets (incl. subsets) held to the same quality standard
  • Criterion/"ground truth" has a clear, defensible, construct-valid definition tied to work analysis
  • Rationale for each predictor/feature (theory or empirics + data quality) obtained
  • Construct-irrelevant proxies for protected characteristics flagged
  • Criterion weighting choices articulated and defended

See also

ai-model-development-audit · ai-model-outputs-audit · ai-fairness-lenses · work-analysis · criterion-related-validation (criterion development, range restriction, concurrent/predictive)

Source: Landers & Behrend (2023), Table 1 (Components 1–2) and "Components Relating to Models" prose.

AI model-development audit (Components 3–5)

Where design becomes a working model. Guidance from the Principles/Standards gets thin here and machine-learning expertise becomes essential — many decisions live only in code (R/Python) or in developers' heads and are highly idiosyncratic to the specific model. The audit's job is to drag those decisions into documentation and test their effect on validity and bias.

Component 3 — Model development

What it is: the specific approach used to refine the model after the initial design. Highly idiosyncratic; the central evaluative question is whether it is documented.

Questions to ask: Once the initial model was created, how was it refined? What approaches were taken, and what likely effect did they have? Is every decision about every model created during development fully documented?

Common failure — selective cross-validation reporting: A frequent issue is the choice among k-fold, holdout, and temporal cross-validation. Performance typically ranks k-fold ≥ holdout ≥ temporal (temporal is worse because model performance decays over time). So developers may report only k-fold results — potentially hiding information or signaling an incomplete development process. Audit move: all three approaches could be run and compared; if they weren't, the reasons for the omissions should be identified. Temporal validation matters most for an operational hiring tool whose population drifts.

Criterion reconceptualization recurs here too: refinement often re-weights the criterion (e.g., task vs. contextual performance). Down-weighting task and up-weighting contextual performance can reduce adverse impact, but alternative weighting schemes carry validity implications (Johnson, 2001; Sackett et al., 2008) and must be explicitly articulated and defended, not quietly tuned for a fairness metric.

Component 4 — Model features

What it is: the feature-engineering process that turns raw input data into the final predictors. The area where ML resources most exceed existing psychological guidance.

Questions to ask: How were raw input data engineered into model features? Was the process conceptually or empirically driven? What alternative feature-engineering approaches were explored?

Apply it (focal example):

  • NLP / bag-of-words: word counts become tokens — entailing hundreds of decisions about the nature of language and its meaningfulness to the prediction. Tokens may be raw frequencies or, alternatively, built from topics. A topic-based model can reduce bias by minimizing the influence of dialect — yet that choice has validity impacts that must be evaluated. Neither is free.
  • Speech-to-text conversion is itself a (often third-party) ML model whose "state of the art" is a constantly moving target, with direct implications for reliability and validity. Audit the quality of the conversion, not just the downstream scorer.
  • Extracted facial/voice features: ask which facial features emerged from analyzing video and what biases were explored in the engineering process (recall the wrinkles/skin-tone/makeup proxies from design).
  • Every step between "raw input data" and "engineered features" must be fully investigated — this is where construct-irrelevant variance is silently introduced.

Component 5 — Model processes

What it is: the specific estimation process by which the final model generates predicted scores (the choice among the many available ML algorithms and how inputs are combined into a score).

Questions to ask: How does the model use inputs to generate scores? How were alternative approaches explored and evaluated?

Apply it (focal example): What stress tests were conducted, and what types of bias were investigated? Did these tests result in changes to the model — and if so, how and why? A model that was never stress-tested for bias, or where tests were run but never acted on, is a finding.

Audit emphases:

  • Remember the bias-variance tradeoff (Lens 3, ML): regularization (ridge/lasso/elastic net) intentionally introduces bias to improve out-of-sample accuracy — desirable, but it sacrifices coefficient interpretability, complicating "why did it score me this way" explanations relevant to procedural/informational justice.
  • Probe whether alternative algorithms were genuinely compared or whether one was chosen by convenience/familiarity.

Pitfalls

  • Accepting k-fold-only performance without asking why holdout/temporal were omitted.
  • Ignoring performance decay over time for an operational tool.
  • Treating speech-to-text / third-party preprocessing as a black box exempt from audit.
  • Letting criterion re-weighting tune a fairness metric without defending the validity cost.
  • Accepting "the model is regularized" without examining interpretability/justice consequences.
  • No documented stress tests, or tests run but never acted upon.

Checklist

  • Every refinement decision documented (or the gaps recorded as findings)
  • Cross-validation strategy examined; k-fold/holdout/temporal compared or omissions justified
  • Performance-decay (temporal) risk assessed for the operational population
  • Criterion re-weighting decisions articulated and defended on validity grounds
  • Feature engineering traced step-by-step from raw input to final features
  • NLP choices (tokens vs. topics) and their bias/validity tradeoffs evaluated
  • Speech-to-text / third-party preprocessing quality audited
  • Estimation algorithm and alternatives evaluated; bias-variance consequences considered
  • Bias stress tests documented, with resulting model changes (or their absence) explained

See also

ai-input-data-and-design-audit (upstream) · ai-model-outputs-audit (downstream) · ai-fairness-lenses (bias-variance, measurement bias) · criterion-related-validation (cross-validation, weighting, corrections)

Source: Landers & Behrend (2023), Table 1 (Components 3–5) and "Components Relating to Models" prose.

AI model-outputs audit (Component 6)

The payoff component: regardless of how the model was built, are its scores good? This is where the AI audit reconnects most directly to classic psychometrics and the SIOP Principles — apply the personnel-selection skills here, on the model's actual operational outputs.

What it is: evaluation of the quality of the predictions the model generates.

Questions to ask: How was prediction quality evaluated — e.g., for psychometric reliability and validity? How was cross-validation conducted, and was it appropriate given the claims about model generalizability?

Apply it (focal example):

  • Reliability — Are scores consistent over time and upon multiple administrations? A candidate who records the same interview twice, or whose video is scored repeatedly, should get stable scores.
  • Validity — Is there evidence the scores reflect the predicted constructs they claim to, and relate to the intended outcome? Don't accept a predictive R² as construct validity.
  • Subgroup differences — Do scores show differences among classes of interest — race, gender, color, national origin, religion, disability, age — and combinations of classes (intersectional subgroups)? AI's high-dimensional inputs make intersectional effects both more likely and easier to overlook.

How to audit outputs (bring the Principles to bear)

  1. Reliability evidence. Identify the conditions you must generalize across (occasions, repeated administrations, alternate prompts, re-scoring) and estimate the matching reliability. For stochastic models or pipelines with random elements, test score stability under re-run. See criterion-related-validation (reliability of measures).
  2. Validity evidence. Treat the model as a selection procedure: assemble criterion-related and/or construct evidence that scores predict the defensibly defined outcome (tie back to the Component-2 criterion audit). Beware overfitting: a cross-validated estimate beats holdout, which beats temporal — confirm the cross-validation matches the generalizability claim (a tool used on future applicants needs evidence that survives temporal validation, not just k-fold). See criterion-related-validation and internal-structure-validation.
  3. Subgroup / bias analysis — run the right test for the right lens. A subgroup mean difference (adverse impact) is a scrutiny trigger, not a verdict of bias (Lens 2/3).
    • Predictive bias / differential prediction — test on the operational score as used (the composite the model outputs), not internal sub-features; ask whether a subgroup is underpredicted, watch power, range restriction, and error-variance homogeneity. See fairness-and-bias-analysis.
    • Measurement bias / invariance & DIF — check whether score meaning is invariant across groups; remember psychometric bias may be expected and "fair" when groups truly differ on the construct (ai-fairness-lenses, Lens 3).
    • Intersectional subgroups — analyze combinations of classes where data and power allow; report what couldn't be analyzed.
  4. Generalizability check. Confirm the validation sample and cross-validation support the population where the tool is deployed (the incumbent-vs-applicant / range-restriction problem from Component 1 resurfaces in the output evidence).

Connecting to the developer's claim

The focal developer claims the algorithm "predicts job performance equally well for all groups using appropriate modeling techniques." That claim is evaluated here — and the paper calls it questionable and in need of audit. Translate it into testable pieces: (a) equal predictive accuracy across groups, (b) no problematic differential prediction, (c) appropriate modeling — and require evidence for each.

Pitfalls

  • Accepting predictive accuracy as evidence of construct validity.
  • Reporting k-fold validity when the deployment claim requires temporal generalization.
  • Testing bias on internal features instead of the operational output score.
  • Treating a mean difference as proof of bias, or "no measurement bias" as proof of fairness.
  • Ignoring intersectional subgroups because each is small (note the limitation explicitly).
  • Skipping score-stability/reliability checks for stochastic pipelines.

Checklist

  • Reliability estimated for the conditions of intended use; re-run/re-score stability tested
  • Validity evidence ties scores to the defensibly defined criterion (not just R²)
  • Cross-validation method matches the generalizability claim (temporal where deployment drifts)
  • Predictive bias tested on the operational output, framed as underprediction, with adequate power
  • Measurement invariance / DIF considered; expected-vs-problematic differences distinguished
  • Subgroup differences examined across protected classes and intersections; gaps reported
  • Developer's "equal prediction for all groups" claim decomposed and each part evidenced

See also

ai-input-data-and-design-audit · ai-model-development-audit · ai-fairness-lenses · criterion-related-validation · internal-structure-validation · fairness-and-bias-analysis · selection-decisions-and-scoring

Source: Landers & Behrend (2023), Table 1 (Component 6).

Sponsors

Details

Marketplace
OpenMatter-Network/agent-io-skills
Source
ai-personnel-assessment
Category
Community

More from OpenMatter-Network/agent-io-skills

All 3