NACE Journal/Summer 2023
Automated video interviews (AVIs) are an emerging recruitment tool that use natural language processing (NLP) and machine learning to screen job candidates. Employers use these systems to evaluate job candidates’ social skills, cognitive skills, and personality. These AVIs are administered remotely and asynchronously—applicants complete the interview on their computer or mobile device at their convenience. Some examples of the outcomes measured are willingness to learn, compassion, drive for results/initiative, and several related to the NACE career readiness competencies.
Two methods power these evaluations: natural language processing (methods for converting natural language text into numbers) and machine learning (statistical models for identifying patterns in data).1 These models are trained on Big Data and can serve as efficient pre-hire assessments. For example, in 2019, Unilever claimed to have already saved 100,000 hours with AVIs–or at least 50 person years.2
Although many early AVIs used paraverbal behavior, i.e., how one’s voice sounds, and nonverbal behavior, e.g., facial expressions, to assess interviewees, the available evidence suggests that including those behaviors for assessment may cause bias and adverse impact.3 Paired with legal concerns about potential bias in the facial recognition software used to measure nonverbal behaviors, most (if not all) AVIs now evaluate interviewees based solely on what they say.4
The promise of this approach is that AVIs apply the same criteria to all interviewees. Unlike face-to-face interviews, interviewees have no ability to ingratiate themselves to an AVI by, for example, talking about the local sports team or complimenting the interviewer. AVIs do not evaluate interviewees differently depending on their appearance. Further, AVIs never get tired, and they are never grumpy from spilling coffee on their shirt or because of personal problems occurring outside of work. Notably, applicants often are not aware of whether their interview will be evaluated by a human or computer: They should explicitly ask so that they can be better prepared to interface with the assessment modality.
Helping Students Prepare
Considering that the success factors in AVIs may differ from face-to-face interviews, it is important for career services practitioners to know how to help their students prepare for these new types of interviews. Fortunately, because AVIs tend to rely on past behavioral questions just like traditional interviews, existing methods—such as the S.T.A.R. method—that most career services practitioners are already familiar with should help.5
Using the S.T.A.R. method provides multiple benefits for interviewees in AVIs. First, students often have difficulty articulating their competencies, and having prepared stories detailed in the S.T.A.R. storytelling method helps address this issue.6 Specifically, interviewees trained in the S.T.A.R. method tend to provide more comprehensive descriptions of the situation, task, actions taken, and results obtained.7 Research has shown that interview answers with more details receive higher scores from humans, and this is likely to be true in AVIs since they are trained to replicate human scores.8 Because traditional-aged college graduates tend to have shorter responses to interview questions than midcareer candidates, it is important that candidates prepare to provide detailed answers. This can be especially important for AVIs, considering their asynchronous nature may cause additional stress and ambiguity for interviewees.
Second, although contemporary AVIs use transformer-based NLP embedding approaches, e.g., bidirectional encoder representations from transformers (BERT), which better capture the style and semantics of speech than earlier NLP methods, such NLP approaches tend to be influenced by word variety, or lexical diversity.9 Thus, interviewees are likely to receive higher AVI scores for having greater lexical diversity.10 We expect that interviewees who more completely describe the S.T.A.R. elements will provide more detailed responses that tend to include greater word variety.
The results section is the climactic destination of a story. Storytellers not acquainted with the S.T.A.R. model sometimes shortchange earlier sections of their story in their rush to the results. If interviewees believe they are simply highlighting problem-solving/critical thinking skills in their answers, they will conclude with accomplishments, goal completion statements, and lessons learned from the experience. However, the results section is an appropriate place to use allusions to make meaning. In other words, if students can describe the story fully enough, they can just point to the results that occurred without explicitly claiming specific knowledge or skills. Some coaches and instructors develop an additional section, an epilogue, to emphasize this opportunity for expounding further.11
Although the S.T.A.R. method is a useful starting point, students often require additional guidance to effectively craft narratives that reflect their knowledge, skills, and development in a positive light. Dialogical Self Theory (DST) and identity construction approaches can help students develop their professional narratives in a systematic way that can improve their chances of passing AVIs. Moreover, given that many career services professionals do not currently use these approaches, training students in these techniques can help career services professionals refine their own skills
Crafting Professional Narratives
Candidates typically use a handful of stories about work and school to illustrate their career agency. Good stories can showcase leadership, group dynamics, and project management. Crafting these stories requires intentional effort.
In an AVI, one’s word choice is the only form of self-promotion, because leading AVI vendors no longer conduct vocal or facial analysis. Instead, AVIs evaluate the aggregate of words used over the span of all responses. Conversely, in face-to-face interviews, candidates’ statements have a different performative quality, because the (mis)alignment of verbal cues with vocal and facial cues can affect the interviewer’s interpretation of the words. Therefore, crafting an effective narrative is especially important in AVIs.
Individuals need a career identity that is emergent and improvised in order to grow and adapt throughout their careers.12 To find a middle ground between uncontrollable economic trends and one’s desired career agency, workers must stir up their imagination, conjure meaning, and develop possible new selves. This process is an iterative, dualistic self-development that plays off society’s structure.13 Similarly, a constructed self-knowledge broadens self-expression. An interview performance incorporating this enrichment is a necessary adaptation to the structure of new recruitment tools. It requires the use of the emotional qualities inherent in reflexive practices. The generated self-knowledge can be used to fill the immediate needs of the AVI and for longer-term career management.
Fortunately, storytelling tends to rely on emotional, rather than cognitive, aspects of life—perhaps because emotion and cognition are inseparable aspects of learning, creativity, and memory.14 For example, when students are asked to describe their most meaningful college experiences, experiential learning is mentioned twice as often as curricular experiences.15 Regardless of the topic, these stories tend to focus on their emotional development and maturity in response to their experiences. Similarly, when reflecting on internships, students tend to focus more on their emotional development and process of adjusting to an unfamiliar work environment and less on the so-called “hard” skills that they learned.16 Research findings in neuroscience also point to the significance of emotion for memory retrieval.17 Considering that both AVIs and human-to-human interviews tend to assess a variety of social and soft skills, this tendency toward emotional valence can be fostered when helping students craft their narratives. Those experiences that stretched their capacity or were personally meaningful to them can be crafted into especially powerful narratives.
Personal Branding Versus Contextualizing
Consider the difference between personal branding and contextualizing as interview strategies. Faced with a human audience, candidates often promote their self-focused attributes and achievements, such as by claiming to be hard-working or good at taking direction.18 However, as interviews become more structured (a characteristic of AVIs), impression management is both less common and less useful.19 Instead of self-promotion, candidates can interpret a recalled scene in their story to add context and feed the algorithm by inputting additional words for evaluation. Metaphorical allusions are appropriate for this task. Metaphors demonstrate both a cognitive and emotional information process.20 They also “express in a succinct manner that which is implicit but is unable to be expressed in discrete, literal language.”21 They are a simple form of symbolism for students to use.
Generating Symbolic Language
Two reflections can engage students and help them generate symbolic language for expressing their narratives. The Career Construction Interview is an exercise that formalizes a student’s role models, favorite stories from print and visual media, and preferred leisure activities.22 This then serves as a foundation for helping them create micronarratives from cultural and societal elements that have a shared meaning. Similarly, the cultural wealth model formalizes group identity attributes.23 Both the Career Construction Interview and the cultural wealth model form concise, common references that add shape to a narrative. Career educators should introduce exercises like these to aid students in deconstructing their career identity. There are many other qualitative career learning approaches for this purpose that could also be considered. Students will be able to draw on these insights to craft each section of the S.T.A.R. model.
In a slightly different vein, a practical use of DST operationalizes meaning making through a reflexive process of identity positioning.24 Students can prepare by creating a multiplicity of voices, all of which represent separate aspects of oneself. When performed in a reflection exercise, this collection of voices helps individuals understand different perspectives of their self-concept. DST is not strictly a narrative approach nor a cognitively based reflection. It is best accomplished through a written exercise.
In DST, there are three types of positions:
“I” positions occur when referring to the self directly. They formalize interests, motivations, and attitudes. This type of statement is also common to interviews when interviewees express distinctions about themselves. Voicing enough of them will expose different dynamics between the statements, enabling students to think divergently about their perspectives and how they can be extended or combined together into a new identity.
Meta positions characterize self-insights and process-related statements. This can help students make sense of their various perspectives and identities, and how they have grown and changed over time.
Promoter positions indicate steps toward action. These help students see how actions they might take can build possible selves and transfer strengths and interests into new roles they aspire to.
Here is a streamlined example of the three positions:
I-position #1: “I can see the nuance in a problem situation.”
I-position #2: “I am good at defining the main issue at hand and finding a way to implement the solution chosen by a group.”
Meta position (encompassing both I-positions): “There are shades of gray in every problem situation. Sometimes, it is unclear how the many concerns can be addressed, and it obscures the path forward. There are always clues to the right direction to pursue if you know where to look.”
Promoter position: “Project management roles for me encompass these strengths.”
A 2016 study explored the ability to think of oneself in a reflexive manner, to increase meta positions and reduce the overuse of “I”-related statements in one’s discourse.25 Learning how to be intentional about meta statements will aid students: They will improve their ability to give insights about how their traits were revealed by aspects of a story they recall, not just declare themselves to have an attribute. Diversifying one’s language with more meta positions generates higher-order phrases and more detailed responses. Specifically, interview statements resulting from meta positions would fit best into task and results sections in the S.T.A.R. model.
Language for the AVI Format
For the practical purpose of creating language for the AVI format, the reflections described here will formalize language that can be used directly in an interview performance. The long-established identity theories are used as tactics, not directly for their original purposes such as locating life themes.
Card sorts such as “Metaphor Making”and “Archetypes in Branding”each have supplemental descriptions. “My Career Story” provides a worksheet to complete. Candidates can practice the use of allusions by interpreting their career construction interview results. Students generate a personalized vocabulary and an improved ability to craft their narrative.
Other reflections aim to instill an ongoing reflexive practice to envision new possible selves. The self-dialogue based on DST, for example, synthesizes and constructs self-concept by flipping one’s perspective back on itself. There are many ways to do this work. For example, in their career writing course, which includes creative, expressive, and reflective writing, Reinekke Lengelle and Frans Meijers lay out several methods. Students learn an adaptive stance for the variety of interview questions they will encounter, helping them to improvise on the spot and learn that a good interview performance is their responsibility.
Preparation Should Be a Central Goal
Just as with face-to-face interviews, adequate preparation in the S.T.A.R. format should be a central goal of candidates in AVIs. Crafting stories increases the number of details provided and the lexical diversity of responses. Career services professionals can help students prepare. Narrative and identity construction exercises are good tools for creating descriptive labels that constitute a personalized vocabulary. These words are likely to be authentic to an individual and, thus, be easier to recall while interviewing. DST and similar creative, expressive, and reflective practices enrich a candidate’s discourse. They help students formalize more dimensions of their experiences to add context to their narratives. Overall, a candidate who uses more emotional properties in their stories will likely increase their word variety, and it will be easier for them to recall their story when experiencing the time pressure that is common to most asynchronous interviews.
Texturizing their responses with these forms of self-knowledge supplements traditional preparation and may improve students’ interview scores.
Alan Jones is the former director of the career center at Notre Dame of Maryland University in Baltimore. He holds a Master of Education in counseling from Northern Illinois University and an M.B.A. from University of Illinois Chicago. He can be reached at aj602312@gmail.com.
Louis Hickman, Ph.D., is an assistant professor of industrial-organizational psychology at Virginia Tech and a Senior Fellow at University of Pennsylvania's Wharton People Analytics. His research focuses on applications of machine learning and artificial intelligence to organizational science and practice, including issues of bias and fairness, assessing psychological constructs from enacted behaviors, helping workers learn and develop, and advancing our understanding of interpersonal perception. Dr. Hickman earned his Ph.D. in industrial and organizational psychology from Purdue University, where he also earned a master’s degree in computer information and technology, and a bachelor’s degree in English.
Endnotes
1 Boyd, R. L. & Schwartz, H. A. (2021). Natural language analysis and the psychology of verbal behavior: The past, present, and future states of the field. Journal of Language and Social Psychology. 40(1), 21-41.
2 Booth, R. (2019, April 19). Unilever saves on recruiters by using AI to assess job interviews. The Guardian. Retrieved from www.theguardian.com/technology/2019/oct/25/unilever-saves-on-recruiters-by-using-ai-to-assess-job-interviews.
3 Booth, B. M., Hickman, L., Subburaj, S. K., Tay, L., Woo, S. E., & D’Mello, S. K. (2021). Bias and Fairness in Multimodal Machine Learning: A Case Study of Automated Video Interviews. Proceedings of the 2021 International Conference on Multimodal Interaction (ICMI ’21). Retrieved from https://doi.org/10.1145/3462244.3479897.
4 Hickman, L., Bosch, N., Ng, V., Saef, R., Tay, L., Woo, S.E. (2022). Automated video interviews personality assessments-Validity, Reliability, Generalizability. Journal of Applied Psychology. 107(8), 1323-1351.
5 Jones, A.K. & Mondragon, N. (2021). Narrative/Constructivism for AI Interviews: A practical application to prepare candidates. Career Convergence Magazine: National Career Development Association. The S.T.A.R. method of interview responding refers to describing the specific Situation in order to set the stage for the story, detail the Task that the interviewee was responsible for completing, the Actions that they took to accomplish the task, and the Results that their actions brought about. Describing the results, whether positive or negative, provides the interviewee the opportunity to discuss lessons learned and other after-event meaning-making.
6 Tross, S. A. & Maurer, T.J. (2008). The effect of coaching interviewees on subsequent interview performance in structured experience-based interviews. Journal of Occupational and Organizational Psychology. 81, 589-605.
7 Goodwin, J. T., Goh, J., Verkoeyen, S., Lithgow, K. (2019). Can students be taught to articulate employability skills? Education + Training. 61(4), 445-460.
8 Murphy, S.A. (2020). Let me tell you about a time: Exploring how storytelling and memory affect behavioral interview performance ratings. (Doctoral dissertation). Wilfred Laurier University.
9 Hickman, L., Thapa, S., Tay, L., Cao, M., & Srinivasan, P. (2022). Text Preprocessing for Text Mining in Organizational Research: Review and Recommendations. Organizational Research Methods, 25(1), 114–146.
10 Jones, et al.
11 Ditewig-Morris, K. (2020). STAR+E: An undergraduate teaching strategy to make the STAR model stick. National Association of Colleges and Employers. https://community.naceweb.org/blogs/kate-ditewig-morris1/2020/12/08/stare-an-undergraduate-teaching-strategy-to-make-t .
12 Muijen, H., Lengelle, R., Meijers, F., & Wardekker, W. (2018). The role of imagination in emergent career agency. Australian Journal of Career Development. 27(2), 88-98.
13 Cohen, L. & Duberly, J. (2021). Making Sense of Our Working Lives: The concept of the career imagination. Organization Theory. 2, 1-15.
14 Immordino-Yang, M. E., & Damasio, A. (2007). We Feel, Therefore We Learn: The Relevance of Affective and Social Neuroscience to Education. Mind, Brain, and Education. 1(1), 3-10.
15 Lackner, C. & Martini, T. (2017). Helping students succeed at employment interviews: The role of self-reflection in e-portfolios. Teaching and Learning Inquiry. 5(2), 3-15.
16 Eden, S. (2014). Out of the comfort zone: Enhancing Work-Based Learning About Employability Through Student Reflection on Work Placements. Journal of Geography in Higher Education. 38(2). 266-276.
17 Phelps, E.A. & Sharot, T. (2008). How (and Why) Emotion Enhances the Subjective Sense of Recollection. Current Directions in Psychological Science, 17(2), 148-152.
18 Stevens, C. K. & Kristof, A. L. (1995). Making the Right Impression: A Field Study of Applicant Impression Management During Job Interviews. Journal of Applied Psychology. 80, 587-606.
19 Barrick, M.R., Shaffer, J.A., DeGrassi, S.W. (2009). What You See May Not Be What You Get: Relationships Among Self-presentation Tactics and Ratings of Interview and Job Performance. Journal of Applied Psychology. 94, 1394-1411.
20 Getz, I. & Lubart, T.J. (2000). An emotional-experiential perspective on creative symbolic-metaphorical processes. Consciousness & Emotion. 1(2), 283-312.
21 Ortony, A. (1975). Why Metaphors Are Necessary and Not Just Nice. Educational Theory. 25, 45-53.
22 Savickas, M. L. (2005). The Theory and Practice of Career Construction. In S.D. Brown & R.W. Lent (Eds.), Career Development and counseling: Putting Theory and Research to Work. (pp. 42-70). John Wiley & Sons.
23 Yosso, T.J. (2005). Whose culture has capital? A critical race theory discussion of community cultural wealth. Race Ethnicity and Education.8(1), 69-91.
24 Lengelle, R., Meijers, F., Hughes, D. (2016). Creative Writing for Life Design. Reflexivity, Metaphor, and Change Processes Through Narrative. Journal of Vocational Behavior. 97, 60-67.
25 Lengelle, et al (2016).
References
Amundson, N. E. (2010). Metaphor Making. Vancouver, BC.
Bolton, G. (2010). Reflective Practice: Writing & Professional Development (Third edition). London: Sage Publications.
Hartwell & Chen. (2012). Archetypes in Branding. Cincinnati, OH. How Books.
Hermans, H., & Hermans-Konopka, A. (2010). Dialogical Self Theory: Positioning and Counter-Positioning in a Globalizing Society. Cambridge University Press.
Hunt, C. (2010). Exploring Career Identities Through Creative Writing. Career Research & Development. NICEC Journal. 23, 18-19.
Lengelle, R. & Meijers, F. (2015). Career Writing: Creative, Expressive, and Reflective Approaches to Narrative Learning and Guidance. The Canadian Journal of Career Development. 14(1), 19-31.
Savickas, S. (2019). My Career Story Universal: An Autobiographical Workbook for life-career success. Retrieved from http://vocopher.com/mcsu/MCS_26_GIUGNO%20(1).pdf.