Lucky Winner: Using a Lottery System to Hire Student Staff [Webinar Recording]

Deandre Taylor:
Good afternoon, everyone. It is so nice to have everyone on the Zoom this afternoon. We have a great webinar that’s going to be happening today. We are really excited to hear from our friends from University of North Texas and other areas around the country. And so with that, I just want to give a few introductions and to thank everyone. I’m Deandre Taylor. I’m the chair for the Commission for Housing and Residential Life, part of ACPA, and we are excited to have our second webinar for our fall webinar series. You can also go to our website, and you can see the list of our other webinars that will be occurring throughout the fall semester.
Here is a shameless plug as well. Commission Awards are open, so if you or someone else on your team would like to nominate a colleague for a Commission Award, particularly for CHRL, please do that as well. Without further ado, I will turn it over to… Oh, before I do that, I’m so sorry, I also want to thank Roompact for being one of the sponsors for this webinar as well. So thank you Roompact and the great work that you all are doing. Now I will turn it over to Dr. Tomas Sanchez, Amanda Vaughn, Shirley Albiter, and Julie Townley.

Amanda Vaughn:
Thank you. We are going to go ahead and get started with our Lucky Winner: Using a Lottery System to Hire Student Staff.

Julie Townley:
Hey, everyone. We just want to thank you for joining today. This is space to acknowledge where you are joining from. I’m Julie Townley. I am a staff member at UNC Asheville. I reside in Asheville, North Carolina on the traditional Cherokee Nation lands, and I’ll let Dr. Sanchez and Amanda…

Tomás Sanchez:
Sure. My name is Tomas Sanchez. Amanda and I reside in Denton, Texas, which is the traditional indigenous homeland of the Wichita and Kadu affiliated tribes.

Julie Townley:
By acknowledging the land and in recognition of modern and historical settler colonialism, including that perpetrated by our North American institutions of higher education, ACPA actively commits to supporting higher ed in decolonizing their practice in scholarship through our mission values and the strategic imperative for racial justice and decolonization.

Tomás Sanchez:
And to introduce more formally who we are, I’ll go ahead and start and then go to my right on the screen. I’m Tomas Sanchez. I’m the Director of Residence Life here at the University of North Texas in Denton, Texas, and I use he/him/his pronouns.

Amanda Vaughn:
I’m Amanda Vaughn. I am one of the Associate Directors of Residence Life at the University of North Texas, and I oversee our hiring and selection process. I use the she pronouns.

Julie Townley:
My name is Julie Townley. I use she/hers. I did work at the University of North Texas with these great folks on the Selection and Hiring Committee. I now work at the University of North Carolina Asheville as a residential care coordinator.

Tomás Sanchez:
And then one of our former presenters is no longer with us in this institution and couldn’t make it. Shirley Albiter was a community director with us using the she series of pronouns. Again, we wish she could join us, but she can’t unfortunately.

Julie Townley:
Awesome. So quickly today, just some learning outcomes for the presentation. You all as attendees will review our implementation of the lottery process for student staff recruitment, selection, and placement. We’ll engage in some dialogue around bias in the hiring processes and how this lottery system created a more inclusive student staff group. And then you’ll learn of the department’s results of assessment, plans for future assessment, changes that have already been made and then we’ll continue to make based on the lessons learned.

Amanda Vaughn:
With the lottery system, what we did was we wanted to combat inequitable hiring practices. So a little bit about our campus climate and our student demographics is that we are a Hispanic-serving and a minority-serving institution. We have a large percentage of first generation college students as well. One of the other things was accessibility, so career preparation and then their ability and then bias, the identity and the professionalism that goes into typically when you’re hiring that we all have, those hidden bias that we have, and then asking the question, what makes a good residence life advisor? Does an interview make a good RLA? That’s the acronym that we use for our positions. But is that really what we need to be doing? And also, again, what are we looking for?

Julie Townley:
You can talk about what you currently have in place in your department to try and reduce some of that bias. Some of the things that we found were, as Amanda talked about, students saying, “I’ve never done this before, so I was not coached on this” or “I just struggle in interviews in general” or maybe “I know in an interview you’re supposed to wear really professional clothing. I can’t afford that, so I’m not going to go to the interview,” or anything like that. So what do you currently have in place in your department to reduce bias from your student staff hiring process?
And then in what ways do you feel like your department needs to improve your student staff hiring process? One of the things most likely is some people are still maybe using the group process or maybe days of interviews or anything like that. So anything you all can think about ways that you could improve. So feel free to unmute yourself, put in the chat, and we can respond from there as well.

Tomás Sanchez:
Seeing a couple of things coming to the chat right now. Just want to acknowledge that. So doing things like telling applicants to dress in what makes them feel most comfortable, what they felt is best for interviews, providing training on implicit bias, working before the hiring process starts. I think a lot of us are probably familiar with that, and a good thumbs up on that too.
And anything else that folks feel like you’ve implemented or cut back on any bias in the process or may want to improve, which is maybe why you’re even here is what I would guess. So more bias training and then also providing the questions ahead of time. It’s a great practice that I think we’ve started seeing more at the professional levels as well too. Keeping stuff virtual to offer equitable access. And again, posting in the Zoom chat.

Julie Townley:
I saw trying to improve to find a better way to review all the information, so having a ton of qualitative data and how to best go through that. That’s super fair.

Tomás Sanchez:
Yeah, our process, which we’ll talk about in a bit, probably still deals with a lot of qualitative data, but I know that’s something Amanda continues to work on and try to address as best as we can, but people are qualitative. Hopefully.

Julie Townley:
Well, feel free to keep putting in the chat, and we can post from there. Obviously, if one of us is not speaking, we can respond to things and then any discourse you would like to have. One of my favorite part of attending webinars is the discourse and meeting people in the chat, so feel free to continue discussing in there.
I’m going to go a little bit more into what this lottery process is, and I’m going to do that through a video. I’ll give you a brief background before I play the video for folks. We should have the audio available to people as well too. But I came across this video and this website of Democracy in practice through a podcast called Revisionist History. This lottery style way of selecting folks democratically was what brought this all to my attention, especially in the early days of COVID, that first year, year and a half, or so where maybe some of us were more isolated than we probably should have been. I know I was hanging out in a building with the same three or four other people when it was usually 30 or 40 people. So this gave me an opportunity to reflect on things, and this video is one of those ways that I started to think about, “Well, what are we really doing here?” So I’m going to go ahead and play it for y’all, and then I’ll talk a little bit more about how that informs what we’re doing now.

Speaker 5:
All around the world, schools are tasked with forming the next generation of citizens and leaders. But student government, the most obvious tool for this, tends to exclude shyer and less popular students and provide a rather empty experience. So we’ve reinvented student government, empowering all types of students to step up and develop real leadership. The key is replacing student elections with lotteries. So instead of a popularity contest between a few ambitious candidates campaigning for votes, students of all types volunteer for the lottery and are called upon to serve. This is a practice that was central to democracy in Ancient Greece, and it makes student government far more inclusive and representative of the student body.
Then after the lottery, we support the student government, empowering it to go beyond organizing dances to address real issues in their school and their community; work that’s challenging but deeply rewarding. And we make sure that along the way responsibilities are rotated so that each and every student representative develops facilitation, public speaking, and other important leadership skills. This approach, centered on lotteries, has shown us just how much leadership potential can be unlocked in the countless students who would never run in an election, let alone win one. It’s shown students that true leadership isn’t about popularity or playing politics, it’s about service and teamwork. And it’s taught them that leadership comes in different forms and can come from each and every one of them. It’s time to reimagine civic and leadership education in your school. Get started at democracyinpractice.org.

Tomás Sanchez:
So, ran across this video and this process through a podcast, again, called Revisionist History. From that podcast, through this video, getting a little bit more from who they are as an organization, came across some tenants that seemed pretty important in thinking about a lottery process and how we could apply it to our RLA position. So I’m going to talk about that just a little bit. I also want to let you know that while all this RLA stuff was going on, we also pretty heads-on went into doing this with our hall councils, our hall governments. That is probably the truest form of a lottery process that we have right now, where pretty similarly, as you saw in the video, folks electronically sign up for a position as we go through summer orientation the first few weeks of the semester, and then they are selected via an electronic [inaudible 00:13:13] that way. That’s not what we are doing with the RLA process, but a lot of those same tenets apply to how we are doing our certification process.
So why a lottery process? As you heard the presenters saying in that short video, it’s an open process that encourages participation. We have a lot of barriers in place for folks. We talked about some of those things already around clothing, Zoom, interview questions, things like that, that in the past, and I’ll say especially pre-COVID, as we’re moving on through COVID now still, that may have presented… that did prevent folks, excuse me, from wanting to apply for positions and especially at a student staff level in which you have 18 to 19-year-olds who are still developmentally trying to figure out their stuff. That may have been the first position that they ever applied for was the ones that we offer here on our various campuses.
As it’s said in the video and one of our questions that Amanda mentioned up front, does an interview make for a good ROA? What I would probably say is that there’s a big difference in running for office versus running an office. The folks who are maybe good RAs or ROAs may not have been always necessarily the best folks that went through the process but yet still ended up with us in some capacity. There’s a question at the end that you saw in the video that is asked of all these student leaders and it’s essentially, “Would you have run if we had done things the old way?” For them that was elections, for some of our campuses, for ROAs, that means interviews, essay applications, background checks, references. And as you can probably guess, most folks tend to say no, they wouldn’t have done that. And so an open lottery process encourages participation.
The other thing that was important to us, as Amanda mentioned again with our HSI, MSI, and first generation status, we have about 42 to 43% first gen students that come to UNT, was thinking about the backgrounds of folks that we bring to the positions. I have no other way to name this except for after the fact. After we did our first lottery process, I went into the room where we saw all 100 and, what is it, 60-something ROAs. What I saw were not the folks that looked like our student leaders in the years previously. If you’re someone who’s familiar with the 1980s or may have grown up in that time period, maybe you can think about the Breakfast Club, the jocks, the nerds, the all those sorts of things. What I saw was a wider range of personality types, clothing styles, hairstyles, how folks carry themselves, body types, et cetera. And that’s all separate from just our demographics that we would consider around race, gender, sexual orientation, et cetera, et cetera. But just how folks even just presented themselves, that looked different.
And so we really wanted to then reflect that broadening of leadership types here within our Ras. Connected to that as well too, we consider ourselves a strengths-based organization. All of our student leaders go through and get their strengths through StrengthsQuest. Amanda, the other associate directors that we have working with us, along a couple of other folks who are all StrengthsQuest leaders, trainers… I always forget the term, Amanda, you can correct me, but we talk about our strengths. And if we really are a strengths-based organization, then everybody has something to offer. So broadening our circles and what those kind of strengths are that we have within an organization is probably then going to be more reflective of how our general on-campus population looks as well too.
To give another example of how that might play out, where I’ve seen it in years past, if you ever did True Colors, you can all recall True Colors based off of Myers-Briggs types, what I saw for many, many, many years at previous institutions were Myers-Briggs types who tended to be blue folks, which tended to talk more about feelings and motions. And then after that it was the golds, the oranges, and then the greens. So even thinking about it from that perspective is, how do you broaden either your strengths, your True Colors, your Myers-Briggs types so that it’s more reflective of what you actually have of your campus population and isn’t just seeking a specific type of student? Because if you’re just seeking a specific type of student, then you might only be benefiting that type of student as well too.
The other lesson in this, going through this podcast, going through this video, digging into the literature a little bit, was that nobody knows anything. We do not know as much as we think or claim to think that we know. And inherently, our measures are flawed, and that is a tough pill for us to swallow within institutions of higher education and learning, is that we are more often than not maybe not as good decision-makers about who it is that we select for things. You can really broaden that not just to positions like this, but to things around elections, who we decide to put in leadership positions, even if we’re not the hiring person, that trying to go off of a process in which we maybe see folks for an hour to several hours for a position that is going to have them for at least nine months here on campus is maybe going to have some flaws within it in our decision-making process.
An example of this that was named in the podcast was the National Institute of Health Research. I think they were called the power brokers of national health within the sciences. They basically have more than several million dollars worth of grants that they would give out to various researchers institutions. For years they had this process, 75 years, they had this process where folks would apply to receive this grant. They’d get vetted out through two different processes. And then these research folks who receive the grants get the monies to run those things, publish, et cetera, et cetera. Well, finally somebody asked the question was, “Well, are the folks who are getting these grants who are then being published, are they actually the ones then that are being cited more than the folks who did not get the grants who we decided weren’t the best, or good enough I should say, to receive these millions and millions worth of dollars?”
What the research showed for them was that no, it was about the same. Whether or not you received these grants and got all this money or whether or not didn’t, the number of times that your research was cited was basically the same. And so that showed that there’s just as many chances that somebody would not be cited after receiving this grant just as much as there is a chance of you not being cited if you hadn’t received this grant. And so their measurements were flawed, and so they switched over to a model in which they did some vetting. They cut out some of the obvious red flags within the process, had a cutoff score of some sort for who could be eligible to receive these grants. And then after that, they lotteried folks. What seems to be the case is that folks are, again, just as being cited often as before, but then that is at least as good as the process they’re going through before as well too.
I would argue it’s probably better. They haven’t updated that research in a while to see if that’s a change or not. But it’s cut back some of, hopefully, their biases that they have as well too. And then my big thing around this, which was apparently a Yogi Berra quote back in the day is, prediction is very difficult, especially if it’s about the future. I would ask you to reflect for a moment about if you’ve had the chance to hire folks, to hire student staff members, to think about how many times have you brought on somebody who during the interview process showed up as wonderful, one of the highest ranking folks and then didn’t perform as high as you thought they would, were just all right, or maybe in worst case scenarios were terrible. And then I’d ask you to think about the other end of that spectrum as well too, where you got somebody, they were an alternate in your pool, you had to go out and find somebody because you needed those mail spaces [inaudible 00:21:57] to be filled, and when you brought them on, they were great, or maybe they’re just fine, or maybe they were just not as great.
But those chances of those things happening on both ends of the spectrum where you’re the highest score or you had to hire somebody just down the block that you could find, one of res halls, was most likely not pretty much the same chance of working it. So that’s this philosophical model we’re thinking about as we move forward. Amanda and her group took that directive for me, which I will take blame and responsibility for, to say, “Hey, this is how we should be looking at the ROA process and integrating a lottery-style approach with sortition in mind.” Amanda will talk about what that sortition process looks like.

Amanda Vaughn:
All right. After listening to the podcast myself and having a lot of conversations with Tomas of, “Wait a minute, what are we doing, and why are we doing this, and how are we going to do this?” and then going to my committee and saying, “Okay, this is what we’re doing. Here’s some snippets from the podcast. How do we figure out what we need to fix, move, and then what can we keep the same that we’ve been doing that makes sense for us as an institution?” And so in implementing, we obviously knew that we were still going to have an application process. I took over this role about seven and a half years ago now, and we’ve been requiring the informational session prior to being able to apply. There’s a lot of information as all you know. Sometimes people get into the job and they’re like, “Whoa, this is not what I thought it was going to be.”
Still sometimes that happens, but we share a lot of information in our informational sessions. And again, they’re required, and they sign in. And then we use Qualtrics, which is survey software for our demographic information and our essay questions. What we did was we took questions from our in-person, and when we pivoted to virtual and after COVID that first year, we stayed virtual because it worked. And so we took all of the questions that we were using and the committee went through all of them and they initially came down to a list of 20, then we moved it back down to 10, and we’ve changed a couple questions these past two years. And so that’s where our essay questions come from, is there are questions we would’ve been asking in person or virtual for that component of the interview.
We also still do an initial review and check. So everybody still gets a grade check, everybody still gets a conduct check, they still have to meet our requirements and that. Our Residence Life staff members are asked to assist with scoring, which is a rubric that Julie will share in a little bit. So we still use a rubric to go through the questions, just like we would use the rubric for the in-person. We had to do a little bit more spelling out of what we expected with the rubric since everybody was grading individually. And then I have a selection committee that in the past has only consisted of community directors and assistant community directors. This year we have community directors, assistant community directors, and because we have a couple other positions, we have housing ambassadors and facility assistants. Some of their supervisors are now on the committee as well as some of our full-time desk folks that interact with these student staff on a regular basis are now on the committee as well.
So the committee members will remove applicants who didn’t meet the minimum requirement. And then after that is done, everybody who’s left has qualified for the position. So they’ve met the score, they’ve got the GPA, they’ve got the conduct, they attended obviously the session, and then we do selection and placement. The committee hosts a lottery days, that’s what we call it. The first year we did it, we thought it was going to take much longer, and it actually did not take as long as we thought it was going to take. I think we were just all a little stressed about how that was going to work. And then if offered a position, the new RLAs must attend an orientation. We do an in-person orientation to finally meet everybody on that level and then also go through some more information about the positions.
And then the placement is also randomized and reviewed. We have a lot of, as I’m sure you all do, we have rooms that are gender-based and then we have our fluid rooms, and so we go through and that’s how we also figure out what our number is for how many people we need to lottery for each position. And then we also have some specialty positions within our living learning communities. And then we have a high school program that’s on our campus, and so we have to hire for that program as well. They have student staff. Like I said, we’ve taken what we think are our top 10 questions that get us the most information and get to know the student staff. One of the questions, and we have a QR code that you can pull up all our questions that we’re using, but we do a conflict style quiz as well that we want to see where the students that are applying what is they’re conflict style. And so this is what we did to implement the process. The next part, Julie’s going to talk about our rubric and share one of the questions with you.

Julie Townley:
When we were thinking about adding questions, so I was at UNT the first year that we did this, one of the things that we were talking about was not only just the question, but when people are scoring, if this is how we are moving people forward or not moving people forward, the scoring has to be really specific and really on par with what we would want to see. And so I included an example of a question. In the past when we’ve done this presentation, we had older questions on here, but this is one for this coming year. You can see that we put check boxes. So each question is worth a certain amount of points. This is what we’re looking for. And so folks who are scoring check or don’t check and add up points based on what is shared.
The hope with that is that people’s then innate bias doesn’t come out when they’re scoring, because that’s one of the other things… I have a K-12 background, and I always heard students say, “If I wrote the same essay with this teacher, I would’ve gotten an A,” or something like that. So we know that that exists as well. There are multiple reasons why people write or don’t write the way that they might. We have the checklist for scoring, which is really important to get everyone on the same page and ensure that the right folks are moving forward based on if anybody were to score.
On the left, this is, I think, the very original rubric. We should update that. But this is the very original rubric. Again, K-12 background love rubrics, and I worked with Amanda who lets people just whatever they’re good at lets them do it. So shout-out to Amanda. But on the left was the generic type of question, so again, that can be updated. But then we have the scoring for each of those. The rubric then that’s the start of it. It talks about how they answer the question, things that we were looking for in the question, but then it goes into the final rubric of scoring, so adding everything up and then moving folks on from there. We are not gatekeepers here, and so please feel free to scan the QR code for all the application questions that we have. Feel free to take what you like, leave what you don’t, but wanted to offer it.

Amanda Vaughn:
Yes, Amanda, I see your question. If you email at the end, our contact information will be there, I will gladly share the new rubric with you, for sure. The next part we’re going to talk about some of the assessment that we have done. What I will say is, when we did this presentation in March, our percentage from the past year staff, it provided us with a diverse staff that better represents our student population. We were at 79% for this staff. This year we’re at about 81%. And so that encompasses every student that are population of what lives on campus and what attends our school. We have a good amount of international students and every walk in between. Some of the feedback that we’ve received from our professional staff is they did say that they felt that they were unable to familiarize themselves with the candidate even though it was bias-free. They needed a little bit more buy-in.
This is some of the feedback initially from our first year, and I’ve got some feedback that we’ve recently gotten from them, but they were overall satisfied with the lottery and the randomizing of the staff takes some weight off of them picking their own staff and then it may be not working out for them. And so when I first inherited this process here, it was a draft-style process. So there was percentages based on the needs of how large your staff was, and then people got to go through. What I was seeing then was that staff were picking for themselves, and there’s some bias in that in, “I need this person to work for me.” But then they leave or there’s an open building and they request to move buildings and they want to take their staff with them. We’re like, “That’s not how this works.”
And so a lot of the conversation is, “We need to hire for our building and our students in that building and that community. And so what are we doing to make sure that that happens?” And so this has actually helped that process. And yes, William, sometimes the best RAs are the midyear RAs for sure. And so, one of the things that I have heard this year is we’re having student staff who are more willing to do the job. The whole staff is not our high achieving student leaders. Our staffs are mixed. You’re still going to have some of those, right, because they get through the process and they get lottery. Just like someone who may be an introvert and wouldn’t have done well in the interview process, but Tomas talked a bit earlier, they’re some of our better RAs or our better student staff, our housing ambassadors or our facility assistance. We need that mix.
I’m an extrovert by nature, and you don’t want 15 of me on a staff. And so you want to have that better mix. And that is the feedback that we’ve received from the staff is that it seems to be a better mix. And oddly enough, because it’s randomized. I don’t know anything about their personalities, I’m going off their scores. I’m going off of, do we have LLCs that need to be filled academic or interest ones? Do we have people who are interested in working with our high school? Do we have people who are interested in working with their hands and our facility assistant program? And so what is that, right?
We’re continuing to do assessment. Every year we ask for feedback. Every year we changed some of the interview questions because we’re like, “Well, that really didn’t give us information that we thought would be helpful. Everybody scored a four.” Not that we want to take away everybody getting a perfect score on a question, but if we’re really trying to figure out who these people are. And once we lottery them, using that to be able to do the placements, we need more and we need to constantly be evaluating our questions. And so we review our new hires reflections of their RLA position. So we have been asking our new hires. We have another associate director that does our training, and there’s an ongoing training in your first semester and so she’s been talking through like, “Is this position what you thought it was going to be? Tell us what you maybe we didn’t give you information wise whether that was through training or through the intersession process.”
And then we do have midyear and end of the year evaluations, and so we’ll share a little bit about that data in a little bit, and then consistently doing that follow-up session with professional staff and not just doing it once. So constantly asking for feedback. My committee, again, consists of a lot of people on it, there’s 10 of us this year, and so there’s a lot of feedback that is coming through that of what they experienced. Our housing ambassador position has been an interview-based position, but then we do lottery them a little bit. This will be the last year that they do an interview. Moving forward, they’re only going to be lottery as long as they meet the standard. So we’re going to remove that part from our housing ambassador position.
The next slide shares our assessment of our end of the year. So from our spring, and again, we all know we’re human, and so we pulled the numbers of the Ras, this specifically is just RAs not our full ROA staff, and so we are missing a few, but that’s just because they don’t get submitted on time. But we have our spring, which that was our traditional still hire. So none of those staff members that year were lotteried. We did 122 of them, 10 of them, which was 8%, exceeded expectations, 110, which was 90%, met expectations, and then 2% fell below expectations. The spring 2023 evaluation, which was our first group lottery, 115 reviews were turned in, so our percentage went up to 26 that were exceeding expectations. Our meet expectations went to 72%, but as you see, our big bump with the exceeds makes up for that. And then our below expectations stayed the same. So that did not change. There was two people that that’s what it was. And then our meet expectations went down just a little bit, but our exceeds clearly went up.
And so we’re going to continue to do this. We’ll have this year’s as well. Eventually we won’t have anybody who hasn’t been lotteried on our staff. This will use another form of assessment, but this one right now is the easiest, and we can pull who continually is just a traditional staff member because we know based on when they were hired.
Some of the lessons that we learned, which is the next slide, is raising the cutoff score. What we noticed the first year when Julie was still here is that we had a really low score because we weren’t sure. We had two options, and we went to the associate director team and we were like, “Because this is new for us, what percentage are we looking for? Are we looking for a C average? Are we looking for a B average? What are we looking for in the academic world of what the percentage means?” One of the things that we did this past year is that we did raise the cutoff. What happened though is that we think we raised it a little too high. And unfortunately in doing that, we ended up removing more male-identified candidates, which intends to be our low amount of candidates that apply in general. And so we’ve already revisited that, and that’s where we’re now doing a 20 out of 32 for this next coming year.
There’s no half points in any of our rubrics. So if they score a 19 or below, they’re going to automatically be removed from the process. And so, one of the other lessons that we’ve decided to do this year is we’re going to email out our questions. We use Qualtrics, so when we email out the application for them, we’re attaching a PDF with all the questions that they’re going to have to answer. Because what we learned last year was that there was a lot of incomplete applications because people were going and clicking on it, starting their questions, and then they were like, “Oh, I need some more time to think about that one.” Then they were leaving that one open. It would get closed out after so long because that’s what Qualtrics does, and then we’d have all these incomplete applications and students weren’t sure if they finished an application or not.
And so this year we’re sending out APDF with all the questions that they need to answer. We’ve explained to them in the informational session that we would suggest that they take those questions in the email, they write out their answers, then they log into Qualtrics, click on the link, and fill it out, copy and paste from Word. This way they can do some spell checking. They can have someone proofread it if they want. Another thing that has come up this year, and I’m sure is a hot topic for y’all at your campuses, is AI and ChatGPT and all of that kind of stuff. And so we have informed them that they can use Grammarly or Hemingway to check punctuation, flow, all that kind of stuff, even though we’re not English majors, so we’re not going to grade on that. But we do have some software that we can check and run papers through to see if they’re AI generated and that kind of stuff, and we are going to use that, but that’s only if Julie’s reading an application and she’s like, “This just doesn’t sound like something someone would’ve written on their own.”
I mean, I’ve tried ChatGPT just because I want to see what it puts out because I’m curious if the students are using it. And so I’ve seen some of the stuff and you can tell. But then also some of it I’m like, “Well, someone probably could have written that,” and so we’re going to have to navigate those waters and I’m sure we’ll have a lesson learned from this next year as well. To make it clear with Julie’s rubrics that she was talking about, we do not grade on grammar, we do not kick out an application if they’re missing a semicolon or if they’re missing a punctuation piece. We do recommend that they get stuff proofread. Go talk to a faculty member, go have a friend read it, read it out loud. I tell people to read it out loud all the time because that’s what I have to do with my stuff, is read it out loud, make sure it makes sense and it flows right.
So those are some of the lessons that we have learned. And again, this is an ongoing learning process. One, we’re at an educational institute, and so I am not one to be willing to… If we need to change something, we change something for the next year. If my committee at the end of the spring semester is like, “Ooh, that didn’t go so well, and here’s what I’m seeing from the backend,” then let’s put it in the notes, let’s make the changes for this next year, so that way whoever I get on my committee the next year can see that feedback. And then we collect feedback again throughout the course of the summer when people are less busy and have more time to think about what they are needing to do.
And then again, a question really for everybody, is there any process that’s truly unbiased? Because even when we’re reading things, you can be biased. One of the things that we do is when we share our essay questions and our applications, we remove people’s names from them. So our staff just gets an essay assigned to them with number five, and they don’t know whose it is. So that way they can’t go look up the student, they can’t see if the student lives in their building. We use Maxient for conduct. They don’t go into Maxient, look the student up to see if they already have any issues and then might not grade things properly. And then we do an average score for our rubrics that wasn’t shared in our rubrics thing. I think we need to add that to our presentation, but I just thought about it. We do an average score, so not just one person grade someone’s essays, we have two to three people grading. Because again, we know we have some staff members who are English majors, and they might be a little bit of a harder grader, just clearly, again, unbiased, they’re just reading it and they’re like, “Oh no, I’m not giving them a point for that.” So we try to make sure that we average that out.
Then myself and my grad, if we see any big differences, so if Tomas gave someone a 30, but Julie gave them a 12, my GA and I are going to go back through and look at that person’s essays ourselves and regrade them to see where they’re falling. Because again, we could have just one staff member who’s clicking, clicking, clicking, because again, sometimes that happens, and then Julie could be a little bit harder of a grader or she’s clicking and doing lower scores because she doesn’t want to get caught that she maybe is not reading properly. But most of the time it’s that we just have someone who’s a little bit a harder grader than others.
That is our last slide. We have left some time in our presentation to have y’all ask questions. I do see, William, that you have a question. “Since there is no process that’s truly unbiased, how do you acquire more male-identified candidates to complete the process?” Here’s what I’ve been doing at our intersessions, we have about 30 intersessions that we run in October, so this month is always a little bit busy because we’ve moved our application up a little bit, our students are getting a whole week off for a fall break this year and so we had to adjust our calendar to that, but what I’ve been doing when I’ve been in those sessions is I’ve been encouraging all of those students to encourage their male friends to apply. I have flat-out told them, “We are always low on male candidates. So in general, if you’re a male, do you have a better chance getting in? Absolutely.” I don’t lie to them about it, I say, “Yes, so tell your friends to apply.” And then I see the male candidates in the room that are like, “Oh yeah, I’m going to go tell my friend who maybe didn’t come to this session.” And then I get questions at the end.
We use Microsoft Teams for chats and stuff, and so we have our student staff promote that, but I don’t think there’s necessarily a perfect way that we have figured that out. If anybody else has any things that they’ve done to try to get more male-identified candidates, please feel free to share.

Tomás Sanchez:
I’m seeing a couple of other questions, Amanda. Thank you, folks, for putting them in the chat. “What conversations and/or requirements do you have to meet to accomplish prior to launching this process specifically with offices such as the opportunities, human resources, general legal counsel?”
So far those challenges have not been presented to us from those various offices. When I’ve explained this process with our multicultural students center diversity offices that we have in our campus, they seemed to understand what the approach was and that if the intent was to get a more diverse group of folks involved, then that was something that they were obviously hoping to sponsor it and match.
From any human resources approach on things, from the conversations we’ve had at the student staff level, our folks who work with the career center who are primarily responsible for student staff seem to understand what we’re doing. We are still hiring folks that have met a level of qualification. Whether they’re hired then and now in a fit that is appropriate is up to chance at that point, which is, I think, always the thing that we’ve had before as well too. Whether or not it actually is a good fit is a matter of chance and bias. So this helps in some ways to eliminate that bias perspective, which is ultimately, I think, better from an HR lens of things. But so far, nobody, knock on wood, has brought any specific challenges to that in a way that’s concerning. We ran into some logistics, as with any REL, RLA hiring process around timeline of when we can hire folks on, when they get onboarded and everything, but I think that’s par for the course even if we’ve done our usual process beforehand.

Amanda Vaughn:
To explain how we run our process, and this might answer a few of the questions that I’m seeing in here, is so right now we are recruiting for the 2024/2025 school year. We require that our staff commit to a year-long position. We do have a returner process, and I see that question in there, and I’ll get to that in just a second, but if you’re leaving in December, if you’re graduating, we do not allow you to return to our staff. And so the only time we typically have to hire for the spring, unfortunately, is if we let someone go or someone falls below what we give for probationary period for grades. So we implement this. Now, October, is when all of our intercessions are happening. October 31st is when the application will roll out. They have two weeks to fill in and finish the application. It’ll shut down November 14th. I was thinking Valentine’s Day in my head for some reason because of the 14th, but November 14th at noon.
As Julie might attest, I’m a noon person because if technology stuff goes wrong, someone is still here in the office to help guide that student who might be stressed out that they don’t think they got their application in on time. We do have Qualtrics send them an email saying that it’s completed and in, but as we all know, technology can not be our friend sometimes, and so we want to be here to answer those questions.
So bias. Yes, we did have staff that were not happy, and I’ll admit that I was like, “What am I doing? Why are we doing this?” What did happen is it saved our staff time. We didn’t have a weekend of 300 interviews that our staff had to participate in Friday, Saturday, and Sunday. They got to review applications. We also listened to our staff and tried to figure out, when can we ask you to review applications during a time when you may be not as busy? And so even though our students get the week of Thanksgiving off, we don’t get three of those days off, and our university has designated that week as a no-meeting week if you’re in the office, and so we utilize that time. They’ll review during those three days and then the week after. And so usually that’s a little bit of a slower time for us, knock on wood, in Res Life, and so they have the time.
The most applications I think we’ve given to someone was 10. Otherwise, they’re reviewing six. This year I think our numbers are going to be a little bit higher. It has not stopped anybody from applying, to be honest. I don’t know it. When I say that is, our numbers are still the same. It may stop a couple of people, but our numbers of who’s applying and who’s attending our sessions is still the same. Oh, go ahead. I’m sorry.

Tomás Sanchez:
Go ahead.

Julie Townley:
I wanted to jump in because there’s a couple questions about buy-in, the supervisors thinking about how to supervise this new team. I was a community director at this point, and so last fall was our first group. And so I had my staff for that time, so I was able to supervise them that first semester. I will say that the buy-in was tough amongst community directors. One of the things, and I continue to say this when we talk about it, one of the conversations that I and a couple other folks had to have with our peers was, “If you believe in yourself as a supervisor, this is going to be okay. If you know that you can train them, if you feel supported to then train your staff, if you know that you can follow up on accountability, all of those things, it’s going to be okay.”
There may be outliers and there may be things that don’t go well, but I know as a community director I felt supported to then go and say, “Okay, this person is not doing well.” I specifically remember Amanda saying, “Well, what would you have done before? Holding people accountable is the same process.” So I will say that buy-in was tough just because it was new, people didn’t really understand what we were doing. Us as peers had a bunch of conversations about how we could reframe it.
And then as somebody who then supervised these folks, I really, really enjoyed the fact that they were all coming in on the same level playing field, that there wasn’t this like, “Oh, we all got selected.” They knew they were all in the same boat, and there was that sense of camaraderie amongst them for that. And then also for me, I’ll just speak as a professional, it really helped me think about just my supervision style and I really need to then get to know these folks who I don’t know at all and how can I best supervise them? So truthfully, I think that buy-in was tough, but it worked out really well. And then a lot of folks, myself included, felt really like it was a good challenge. I mean, I feel like it made me grow a lot. So from a community director standpoint, ultimately it’s just that initial anxiety, but it worked out so well, and I think we were all super happy with our staffs.

Tomás Sanchez:
There’s a question in there about our trainings that might help folks feel like better supervisors or that help the RAs just develop more equally on the same page as much as they can throughout the year. I think our strengths approach is really helpful in how we work together as a team but in terms of also how the CDs and ACDs supervise their RAs. They know what everyone’s strengths are, and if you enter things from that perspective, you don’t bring in the deficit model of supervision, then you’re golden. You know that this is the way this person works the best, so let me make sure when I’m giving that feedback that everyone needs at some point it’s from this lens. So I think that’s one thing.
The second thing, as Amanda mentioned earlier, we have another associate director responsible for student staff training and development. The development that she has in place is essentially regular all-staff meetings about once a month or so, and then regular reflection questions that allow folks to say to themselves, “How are we doing?” The new ROAs… excuse me, RAs go also through an extended training throughout their fall semester, so they’re getting that regular development on top of that as well too. Our reflection questions also happen to be the questions that we have for our returner process. So that’s helpful that it’s just not something we’re springing on them at the last moment either. We are truly committed to making sure that they’re reflecting on how they’re doing in the position and if it’s something that they want to come back to, that they feel like, “Yeah, this is where my strengths are. This is how I was happy with my position,” then you’ve answered all the questions already. It’s now just a matter of you taking the responsibility to move forward in that process. Amanda?

Amanda Vaughn:
So speaking of our returner process, how we work that out is our returners don’t go through the lottery again. So they have reflection questions, and then what they do is they turn that into an application. So we do another Qualtrics, they fill it out. They’ve done these questions, they’ve talked to their supervisor about their answers, they can tweak their answers. So if I turn something into Tomas in October 1st, but the application’s not due till December, I can update my answer, maybe add a little bit more. The point is we want the supervisors also to be talking to them about their experience with these questions and the job. And so then what they do is they turn that in and then they get a presentation prompt. So it’s not an interview. They give a presentation. There’s different questions usually every year, we try to change it up because our staff who come back more than one year or into their third year, we want them to do a different presentation and reflect on something different, and so they have about a 15-minute presentation that they give to some of our staff members.
They don’t present to anybody that’s in their area, so my supervision area doesn’t present to each other, they’ll present to a different associate director’s area. And then they’re scored on their essay questions, they’re scored on their presentation. And there’s also a recommendation that goes along with the mid-year evaluation for our staff that gets turned in by their supervisor. There’s a lot of conversation, and that prompts a lot of conversation about, “Are you wanting to come back? Why do you want to come back?” And so we’ve had students, and Julie was here when this happened, where we had some returners that went through, submitted their application, but then pulled themselves out before the presentation. They were like, “You know what? No. After reflecting, I’m going to finish the year out, but this is not the job for me.”
That’s what we want. In our returner process we have only told four people no, and a couple of those people have appealed and one was able to get back in off their appeal and one was not. And so they go through that process as well as with their grades, their conduct, we still check that kind of stuff every semester. So with both processes happening at the same time, what we do is October is our sessions for our new people. November is when their application is due. Our returners application is due in December. And then at the end of December, everybody should be graded between the new people and the returners. And then in January we do grade checks and conduct checks. We have to wait for the university to confirm grades. We do our lottery at the end of January, then we offer positions at the beginning of February. Then we have our orientations. We have a really cool all-staff meeting. And then with this year, what we’ve decided to do is in April we’re going to do a kickoff to summer and a kickoff to training.
So our training associate director and myself will run this meeting, and then I’ll turn it over to pass the torch of all of them to her. And then she’ll let them know what to expect when they come back at the end of July for training. But then we have all these touchpoints without them through the spring semester.

Tomás Sanchez:
And speaking of passing the torch, I believe we’ve gotten the heads-up that time is up. But we wanted to thank you from our team on behalf of our folks who can make it here today for having us. I hope this has sparked some interest in doing something like this, or at least taking the opportunity to really question what is it that we’re doing and are we being intentional about how we’re doing it in order to create access for students at a variety of different levels? So Deandre, I’m going to leave it to you to close this out.

Deandre Taylor:
Thank you all so much for the very informative webinar. Great work. Keep up the great work to make everything equitable as much as we can for our students. Thank you ACPA, and thank you to Roompact. Our next webinar will be November 6th, so make sure that you sign up for that. Other than that, I truly hope that this was informative, and have a great day. And if you all are traveling to Long Beach for RCA, safe travels.