NACE Journal, May 2020

Those of us who develop and deliver career programming to students or alumni will hardly be shocked to hear that incorporating active learning strategies yields benefits. Many of us would even argue that not incorporating some kind of engagement—a short worksheet or perhaps a peer review of job materials—misses an easy opportunity for applied learning. At New York University (NYU), where I develop the bulk of the career programming for current and recent graduate students, we explored the idea of active learning in a much more intensive and immersive kind of program and came up with the Mock Hiring Committee. The Mock Hiring Committee is an extended simulation that allows participants to assume the functions, responsibilities, and perspectives of those in charge of hiring decisions with the purpose of increasing participants’ understanding of the hiring process. Our goal is to transform the way students understand this process and how they view themselves within it.

The Origin of the Idea

To create this program, I wanted to apply a model similar to what I had seen during my years working in college admissions, but had never personally encountered in career development. In my admissions career, my fellow counselors and I sometimes engaged in mock admissions committees. We occasionally led these for community-based educational organizations or individual schools that had organized these for their students and parents. The basic premise was that participants working in small groups would receive a packet of fictional applications along with a profile about a pretend college, including the types of students who were a good fit for the institution. The participants would then give a holistic review of the “applications” and then, as a mock admissions committee, decide who they felt they wanted at their fictional college.

Discussions amongst the members of these committees could be surprisingly intense as people developed strong feelings about who they felt deserved the coveted spot. It was a level of investment and engagement that felt radically different than the many talks I would give about the process of applying to schools and the tips I would share for going about the process. I hypothesized the reasons for this. One possibility was that there was power in participants feeling a sense of ownership in a process for which they often felt they had little control. I also wondered if it was a way for them to engage with the process through a means that generally just felt more resonant. Regardless, it was clear to me then, as it is now, that a simulation made for a markedly different, and seemingly more meaningful, learning experience. This was something I knew I wanted to replicate with career education. So, in the summer of 2018, I set out to create a mock hiring committee for NYU to do exactly that; the program launched in fall 2018.

The Need

There were three factors that drove the creation of the program.

1. Better insight into the hiring process: There is a very large international student population at NYU, and the majority of graduate students come to their programs with three years or less of full-time work experience. As a result, many of the students we serve have limited experience working in the United States and are often unfamiliar with the expectations that employers have when seeking to fill roles. So, the program needed to provide students with a high degree of insight into the process.

2. A one-time, comprehensive program rich in content: As NYU is an urban institution, many of our students commute from across the New York metro area. This makes it challenging for them to attend many shorter programs offered at varying times on individual topics. (Previous attempts to create comprehensive programs spread out over multiple days inevitably saw a steady decline in attendance, meaning that many students would end up missing essential content.) We determined that this program would need to be done in one day and have both breadth and depth.

3. Opportunities for active learning: While we identified content that would be important for greater student insight into the process, we wanted the simulation model of delivery to be informed by the NACE career readiness competencies.¹ Specifically, we wanted content to be taught through extended opportunities for collaboration, leadership, and critical thinking. We also wanted to use digital learning in meaningful ways that complemented our goals; part of that meant testing students’ ability to demonstrate learning, as opposed to only surveying for satisfaction.

Design and Implementation

So, how would all of this come together, and would it actually work?

Launched in November 2018—and offered once each semester since then—the Mock Hiring Committee begins with each student joining a table. Every table becomes its own hiring committee for which students designate a hiring manager and HR manager to represent their interests at various points, while the remaining students are classified as team leads.

With these leadership roles established, the daylong simulation begins. As facilitator, I introduce their purpose: to fill a vacancy for an analyst at the policy institute where they are “employed.” We provide a fairly detailed overview of the fictional organization’s size, mission, corporate culture, global presence, and specialized expertise, and then share the responsibilities for the specific analyst role. Their task is to:

• review the resumes and cover letters of a pool of five fictional applicants and whittle them down to two interviewees;
• watch video interviews of two candidates and determine who will be recommended to receive a job offer; and
• engage in a negotiation with their fictional candidate and determine what counteroffer to make, if any.

Job materials

Once students mutually determine the organization’s priorities for the types of candidates that would be most desirable, they apply those priorities in evaluating five fictional candidates’ resumes and cover letters. We intentionally incorporate some of the most common errors into the design of these materials. For example, some resumes have unappealing formats, fail to show impact, are poorly organized, are overly long, or do not highlight skills relevant to the job. Meanwhile, some cover letters are written with inappropriately informal tones, accidentally name another company, or are not tailored to the job.

Students discuss what they like and do not like with their fellow hiring committee members to determine how they would rank the candidates and select their top two for interviews. Once every team submits its rankings, they can compare with other hiring committees and see which kinds of resumes and cover letters tend to be favored by decision makers.

Following this first portion of the simulation, we briefly discuss best practices for resumes and cover letters to explicitly call out what they have just proven to themselves: There are best practices that, when applied, are generally much more appealing for a reader. We then give students the opportunity to view the same resumes in a large size—we have them blown up—so they can make any needed improvements as a group. It is a striking experience to see how well they apply what they just learned; we have consistently seen groups notice nearly every error at this point.

We ask students to then diagram the cover letters they just reviewed, to identify whether the structure was appropriate and whether the necessary content was present and well-organized.

Next, we conduct a short quiz, which students can complete with their smartphones. The quizzes help us assess whether students are able to answer content-based questions on the material that has been covered, which then enables us to clarify any concepts that have not been sufficiently grasped. The platform also allows students to see who gets the correct answers first, which enables students to compete for prizes; that function makes the testing element much more palatable for students. Overall, aggregated results* from our quiz show that:

• 94 percent of participants correctly answered our question about the types of information that should be included on resumes.
• 91 percent of participants correctly answered our question about the types of information that should be included on cover letters.

Interviews

During the interview portion, students receive lists of interview questions and log into a laptop located at every hiring committee’s table. Laptops come preloaded with links to unpublished YouTube videos. (Note: These are captioned for accessibility and also to cope with the volume of noise that comes with multiple hiring committees watching videos simultaneously in proximity.) Students are instructed to view only the interview videos of the two candidates they selected after the job materials review process.

The videos are scripted to incorporate both common mistakes and best practices, and they are filmed in-house with staff and student workers posing as the interviewees. After watching their chosen candidates, students discuss the strengths and weaknesses they see. Many of the students report that the videos are particularly helpful, since the majority of them have never seen someone interview before. This is often the first time they have seen what a strong interview performance looks and sounds like.

Once students have watched both videos, they make a group determination as to which candidate they want to hire. The participant from each committee deemed to be the hiring manager then presents the group’s recommended candidate to a staff member acting as their division director and has to field questions about the potential hire. This element of the simulation serves as an additional opportunity for students to practice public speaking as they lay out their reasoning for a business decision to an authority figure.

Following this process, the staff facilitator provides a brief overview of interviewing best practices and strategies for answering difficult questions.

Negotiations

The final part of the simulation is the negotiation phase. Students are presented with an employment offer drawn up by HR; the offer includes salary, benefits, and a proposed start date. They are then told their chosen candidate would like to discuss the offer, so they select another pre-loaded video on their laptop, which includes the negotiation with whichever candidate they have chosen. While all but one of the fictional candidates attempt to negotiate their offers, they do so with varying degrees of success, so students must evaluate the arguments they hear. If they are persuaded, they must determine a counteroffer, which the HR manager then presents to other teams.

This process allows students to hear negotiation tactics that are compelling to various degrees; this helps them to determine how they would want to approach such a conversation for themselves. It also enables them to think through which types of things can be negotiated beyond salary. Students are further able to see that, in many cases, there may be a more favorable offer available, even if the candidate does not get everything requested.

Following this exercise, students are debriefed on negotiation strategy by the facilitator. They are then tested. Overall, aggregated results found:

• 97 percent correctly answered our question about interview strategy.
• 90 percent correctly answered our question about when to negotiate.

Student Response

As of this writing, the program has been conducted three times—November 2018, February 2019, and September 2019—with 120 students participating, 86 of whom completed and returned their evaluations. In addition to testing student understanding of career content with virtual quizzes, we wanted to assess the participant experience with this format. The overall results have been overwhelmingly positive for us:

• 100 percent of participants agreed strongly/agreed somewhat that they learned what to include in their resumes and cover letters to make them stand out to employers;
• 100 percent of participants agreed strongly/agreed somewhat that they felt more knowledgeable about preparing for interviews;
• 100 percent of participants agreed strongly/agreed somewhat that they felt more knowledgeable about preparing to negotiate a job offer; and
• 98 percent of participants gave the program an overall rating of excellent or good.

We also sought qualitative feedback about the simulation model for teaching this content. Through this feedback, three main themes emerged: 1) Students felt that seeing an employer’s perspective demystified the process for them; 2) Tangible examples of successful and unsuccessful candidates helped them realize mistakes they had been making, while seeing strong candidates showed them how to correct these mistakes; and 3) They enjoyed being able to make decisions with their peers through interactive discussions and debates.

Additional Outcomes

One of the discussions we had hoped might emerge between participants and facilitators was about assumptions and biases of those reviewing resumes toward the applicants. While this was never intended to be a primary goal of the simulation, we intentionally gave several applicants gender-neutral names on their job materials. In multiple cases, we saw participants express surprise during the interview videos when the perceived gender identity of the candidates was different from the ideas they had formulated about who they were interviewing. In one instance, we also had a participant make assumptions about the age and family responsibilities of one fictional candidate with a Ph.D., leading the participant to assume the candidate was less interested in traveling for the job and therefore less qualified. We used these moments to briefly discuss legal and illegal interview questions and how bias can sometimes influence decision making.

There was a second interesting outcome that emerged during the negotiation activity: When we allowed the participants to make counteroffers, we noticed that a fair number of hiring committees counteroffered with non-monetary incentives, such as tuition remission, virtual work options, and child care grants, as opposed to focusing only on salary. This suggests that Gen Z and young millennials place significant value on work/life balance and growth in terms of the job.

In Theory and Practice

Simulations are a common strategy for teaching content, and with good reason. According to Smith and Boyer, “Simulations have the power to recreate dynamic...processes in the classroom, allowing students to examine the motivations, behavioral constraints, resources, and interactions among institutional actors.”² Active learning, they say, is believed to have five major benefits for students: 1) deeper levels of insight into process, 2) greater attention and activity levels in the process, 3) more long-term retention of information, 4) development of analytical skills through collaboration, and 5) stronger speaking and presentation skills.³

With this knowledge and understanding, we felt compelled to use a simulation model to deliver this content and further felt there were opportunities for students to develop skills through the program, based on the NACE career readiness skills.⁴ Ultimately, we wanted the simulation to translate into the ability for students to apply the material and have it inform their approach to their job search in ways that would provide value.

There is also research about simulations and related active learning strategies that shows participation leads to better knowledge of the material being taught and also better performance when being tested, compared with students who learn through more passive means.⁵ The act of frequent testing itself is widely believed to be a key component of learning and long-term retention, as is relevance for the participant.⁶ As a result, having short testing experiences incorporated strategically into the design was a priority for our purposes at NYU.

To make the process as relevant as possible for our graduate students, we opted to make the vacancy being filled an “analyst” position at a fictional consulting/policy firm. The decision to do that was based largely on the interest in that role we see from our graduate students, but also because the analyst position is inclusive of many different fields of study and tends to prioritize many of the strengths our graduate students have, e.g., teamwork, analytical, and communication skills, and ability to conduct research. In addition, we believed our students had some context for the analyst position and therefore would be able to use that context in their role play as hiring managers.

Challenges

The biggest challenge in designing and implementing this program is the preparation that it takes. Because there are five applicants who students review, a resume and cover letter must be created for each, and an interview and negotiation must be scripted and recorded for each. This takes time and research. Many institutions without dedicated communications teams may also find that partnerships with school TV stations or theater groups can be valuable in the video creation process. (Note: Theater or other acting experience is very helpful in terms of the videos.)

Regarding set up of the actual event, every session requires attention to detail with technology. In addition to every table needing to be fully prepared with all of the materials for participants, it is helpful to have extra backup laptops available and fully loaded with the videos, in case there are issues. Testing the quizzes and videos with staff ahead of time to make sure that any bumps are smoothed out in advance is essential, since it is difficult for a facilitator to troubleshoot for multiple groups. Finally, if possible, it is best to have at least one other staff member on hand to help with day-of event execution.

Another challenge is event size. We found that 25 students seemed to be a highly effective size group. Our September 2019 program saw 70 students participate, which we considered to be room capacity for the set up. While students still reported very positive feedback, it meant that every hiring committee was made up of eight or nine students; this made it somewhat more challenging to ensure that every hiring committee was engaging all of its respective team members fully.

The Future of the Mock Hiring Committee

Overall, we were very pleased with how our simulation has worked to deliver meaningful, relevant content to students in an interactive way. We hoped that students would gain insight into the process from another perspective. We further hoped that this insight would translate into ideas about how to implement potential improvements in their own approach. This turned out to be remarkably consistent with the feedback we saw from participants.

This was the first major effort at the Wasserman Center at NYU to use simulation-based learning, and we have since developed additional simulation-based scenarios for specific populations, including doctoral students and postdocs on the academic job market. Long term, we intend to revisit our other career education programs and use further simulations and case studies to teach essential content and promote peer-to-peer learning.

Endnotes

¹ National Association of Colleges and Employers (NACE). Career Readiness Defined. Retrieved November 22, 2019, from www.naceweb.org/career-readiness/competencies/career-readiness-defined.

² Smith, E., & Boyer, M. (1996). Designing In-Class Simulations. PS: Political Science and Politics, 29(4), 690-694. doi:10.2307/420794.

³ Ibid.

⁴ NACE.

⁵ Bromley, P. (2013). Active Learning Strategies for Diverse Learning Styles: Simulations Are Only One Method. PS: Political Science and Politics, 46(4), 818-819. Retrieved from www.jstor.org/stable/43284771.

⁶ Miller, M. (2011). What College Teachers Should Know About Memory: A Perspective From Cognitive Psychology. College Teaching, 59(3), 117-122. Retrieved from www.jstor.org/stable/41305121.