AI-augmented recruitment interviews : a feasibility study of real-time facial video analysis under recruitment-like conditions

Abstract

PURPOSE: This paper examines whether real-time facial video analysis can function as an interpretable decision-support layer in technology-mediated recruitment interviews. It addresses a gap between research on AI-supported recruitment, which mainly emphasises efficiency and process standardisation, and research on affect-related video analysis, which typically prioritises model performance outside realistic organisational hiring contexts.

DESIGN/METHODOLOGY/APPROACH: The study adopts a staged research design combining: (1) laboratory grounding through comparison of optically derived facial activity traces with EMG-related measures, (2) development of a remote-capable and interview-compatible capture procedure, and (3) prototype evaluation in recruitment-like scenarios. The analytical pipeline combines convolutional neural network-based landmark detection, FLAME-based facial reconstruction, and FACS-consistent descriptors to transform facial activity recorded from standard video into biosignal-like temporal traces. Recruitment-oriented evaluation was conducted on a sample of 75 participants, with system outputs compared against the ratings of a single expert observer.

FINDINGS: The results indicate prototype-level feasibility rather than validated recruitment effectiveness. The strongest agreement with expert assessment was observed for emotionrelated outputs (85%) and stress-related inference (80%), while lower agreement was found for interactional responsiveness (60%) and nonverbal behaviour based on microexpression analysis (55%). Aggregate agreement reached 75%. The prototype also proved operationally feasible within a structured interview workflow, requiring up to 20 seconds of behavioural observation for selected constructs and generating outputs within near-real-time latency. However, the findings do not establish predictive validity for live hiring decisions.

PRACTICAL IMPLICATIONS: The proposed approach may support structured recruiter observation by providing auditable, standardised, and temporally interpretable behavioural indicators during selected interview segments. Its use should remain strictly supportive, human-supervised, and bounded by governance safeguards relating to privacy, consent, transparency, and fairness.

ORIGINALITY/VALUE: The paper contributes a recruitment-oriented framework that links physiological grounding, remote-capable data collection, and prototype-level expertconcordance testing. Its originality lies in treating facial video analysis not as a tool for autonomous candidate judgement, but as a cautious and reviewable analytical layer designed to support, rather than replace, human decision-making in recruitment.