ResEdChat Ep 104: Harnessing Pulse Check Assessments of Student Learning with Chad Lystad

Dustin Ramsdell:
Welcome back everyone, to another wonderful episode of Roompact’s Res Ed Chat Podcast. If you’re new to the show, every episode, we feature a variety of guests, covering a variety of topics of interest to higher ed professionals who work in and with university housing, residence life, residential education, whatever you might call it. But we are having our conversation today. Continuing sort of, the broader trend of topics that we’ve had this year surrounding around assessment, the curricular approach, applying it, and the variety of ways that you can kind of accomplish that. So this one in particular, I think will be really interesting is looking at more quicker applications, more timely insights that you can get to put a little bit of a contrast towards the more longitudinal, broader efforts that I think we maybe more often see.
So I’m excited to explore this and we will get to that topic. But first, as you always do, I suggest, Chad, if you want to introduce yourself, give a brief overview of your professional background, how you got to be where you are today, and then we’ll go get a little bit more in the inception of your pulse check assessment efforts and all that you’ve learned from that.

Chad Lystad:
Great. Thanks, Dustin. I’m happy to be here. My name is, Chad Lystad. I work at Concordia College in Moorhead, Minnesota. I’m the Assistant Dean of Students and Director of Student Engagement here. I guess, a brief overview of my professional background, and I swear some of this will be relevant to what we talk about in a little bit, but my background is in clinical psychology. So, I got my master’s in clinical psychology and I spent several years doing psychology research, quantitative and qualitative research, usually NIH funded, with people experiencing mental health concerns. So, I did a lot of research, a lot of publications and stuff like that, throughout the early part of my career. But then I realized that I missed higher education and I realized, I had the light bulb moment of, wow, I can actually work at a college. So, applied to work at higher education, work in higher education, and over time went from being an academic advisor to working in orientation and then moved into my current role with student engagement.
Higher education is kind the dream job. I really, really love working with this population of students, and it’s just really, really fun to be able to see this growth over time and to utilize some of our assessment pieces that we’re going to talk about to show growth over time, which is really, really fun. So I’m happy to be here and I’m looking forward to our conversation.

Dustin Ramsdell:
Yeah, yeah. I appreciate you joining in and I think it is a good opportunity here because you aren’t, I guess, for lack of a better term, classically trained or brought up through residence life, residential education, but I think we’ve done, I think, and we’ll continue to do episodes that do show the broader scope of the curricular approach and obviously, just student affairs work or student support, student engagement work is broadly applied both in res life departments or student affairs and whatever kind of terminology and stuff. It’s like we’re all on the same team, the sort of co-curricular efforts that are happening, so if folks are scratching their heads or whatever, we usually have a lot of residence life folks, but really appreciative of having other folks, other collaborators and colleagues and folks on, to talk about their learnings and how they do or can apply in the residence life context.
So we are… The string of guests that we’re having that are sort focusing in on assessment and sort of all the different angles on that are coming from the recent institute and curricular approach that Roompact is really proud to be a part of. So, you are a part of that event talking about your real-time assessment efforts, the pulse checks that you’re doing. So, if you want to explain a bit about the inception and creation of these efforts. So, we’ll start there and go along, I guess, the benchmarks of the journey of creating it, doing it, and the learnings that you’ve had. So, how did this come to be?

Chad Lystad:
Concordia College started our curricular approach in 2019, and we have really strong leadership from our Vice President of our division. Our division includes residence life center for student success, holistic health, student engagement, and athletics, and career. And our leadership really, is embracing of the curricular approach and really gave us kind of carte blanche to try things, to have some fun with our strategies, with our assessment, and just really gave us the reins to utilize the curricular approach. So, the way my portion of the curricular approach came into being was, at the time I was working in orientation and first year transition, and I was in consultation with some of my colleagues and we were talking about how to assess this big event that we’re doing, and it kind of came to us and it kind of came to me to, do really quick hit assessments that allow us to change our programming on the fly.
And I’ll go into detail about what it is that we do here, in a moment, but I think the overarching goal of these assessments is, to know in real time what the students are learning and to know in real time what questions they still have. And if we know these things in real time and if we distribute those results, those assessment results widely, then we can be really nimble in our programming and we can be really nimble in the way that we support students.

Dustin Ramsdell:
That does feel like a really strong epiphany to have is that, and it’s not, I guess you can kind of clarify this as a quick follow up, but that idea that it’s not one or the other in the sense of that a lot of surveying or assessment efforts or outreach that you’re doing to get feedback from students might be like, oh, we do our twice a year, once in the spring, once in the fall big survey to try to get as much at once. If we’re doing it, we’re going to try to just really scoop up as much as we can. It’s like, yeah, that’s well and good. That’s fine. I mean, I think sometimes you might struggle to be like, oh my God, we get to spend 15 or 20 minutes filling out all these questions and giving all these thoughtful comments, whatever.
It’s like, maybe you get a few people that do that, a few of these busy students and stuff. But the idea that it’s like, there’s a place for that and there’s also a really productive place for shorter, quicker, timelier things, that you’re gathering and taking action on. So, is that how you see it, that you’re trying to get the best of both worlds of how you’re doing outreach and getting feedback from students?

Chad Lystad:
Yeah. So you said, epiphany, and that really was what it was like for me. Again, I’ll go back to why my background in clinical psychology and research is relevant. At the start of this, I was looking at this through the lens of hard science research, assessment as we used to hours of interviewing and feedback surveys and data collection, because all of it was for publication purposes, and the epiphany for me was one of my colleagues said, this is not a thesis. This is not a doctoral thesis that we’re writing. This is really a way to know if what we’re doing is hitting the mark, and it’s a way of collecting information so that we can tell our story. So it doesn’t have to be something that is worthy of being peer reviewed down the line. So, that was the epiphany to be able to… Yeah, it’s important to collect those larger longitudinal survey information pieces for different purposes, but for our purposes, it’s just the quick hit. It’s the quick hit. What is it that they’re learning right now, and how can we use that information, as we move forward?

Dustin Ramsdell:
When I think, even for something like this, it’s like that, I feel like, is so foundational or should be a fundamental piece to approach this, because it is that idea of we’re going to know more precisely what we learned or didn’t learn or want to learn more about in the moment versus being like, hey, so over the last five months or something, what are all the things that you’ve learned at all the events that you’ve gone to? You might be like, I don’t even remember all the events that I’ve gone to. It’s like, I might just remember the most recent ones, and you’re going to get a artificially or improperly weighted feedback of just like, oh, we sent this out end of the semester in May. We’re going to get a lot of insights from the big thing we did last, back in April.
So, yeah. I think, this is the first time we’re talking about something like this in particular, but I think just wanting to bold and circle and underline highlight, this really should be incorporated in some way. It clicks so easily for me, when you’re saying, I think that epiphany where like, people can get stuck and sort of the rhythms of like, well, yeah, I mean, we just done the surveys in this way and that’s worked out well enough, but the idea that you’re not having to stop doing that to do this other thing that’s just like, well, no, let’s just, especially if a good way of tracking who comes out to certain things or who’s involved.
It’s like, okay, well, let’s just trigger quick email, survey, or notification, or text, or something. Some way to just try to reach out to those students to get it before it’s gone or there’s going to be an expiration date, or sort of it’s going to just start to fade away as time goes on. So, it’s like the quicker we can capture that, the better. It’s just going to be fresher and more relevant and everything. So, I think just to kind of segue off from that, because I guess that’s sort of setting up the benefits of this approach, and I think we can kind of break this apart to just not to give each part it’s due focus here, but what are the benefits of this approach that you’ve seen for your team? I think we’ve kind of started to hit around this, but just integrating this approach, how has it helped your team?

Chad Lystad:
This has helped our team immensely, because it has demonstrated to our team that this can be really easy and it’s taken away the scariness, I guess, of assessment. I think, it’s really nice to see people embrace that assessment can be easy and it can be quick. It also has benefited our team because we can show in real time that the students are actually learning what we say they’re learning, or in some cases, they’re not learning what we say that we hope that they’re learning, so we stop doing that thing or alter it. So having this form of assessment, really allows people to even further buy into the curricular approach.

Dustin Ramsdell:
Yeah. That’s something I didn’t really anticipate, I guess, that idea of the buy-in. We’ve had episodes recently about how we’re implementing an approach like this more broadly and the change management piece. So, I’ve appreciated that we’ve had episodes going into that more fully, but this in particular, sort of demystifying, I guess, I get if it’s one of those things where it’s like, well, we’ve got to like… I guess just to keep using this example, like, oh, the twice-a-year survey we do, there’s so much high stakes on it, or it feels very, I imagine sort of, this big ship that you’re trying to move and adjust or whatever. So I think, yeah, the idea of, you can’t mess anything up too bad if you’re just sending a very short survey about a thing that somebody went to or whatever. So I think, that is a really good thing.
I mean, yeah, we’ve kind of detailed the idea of the insights that you get. You’re able to get information in a very timely fashion, but also take action on it quickly to pivot and make adjustments versus it being like, well, we were operating without it for months, and it’s like, oh, I wish I knew people were like, hey, this time of day is not great for events. It’d be like, that’s why a lot of people don’t show up. We didn’t know that, over the course of the whole semester. So, the other part of this that I want kind of to break apart and have separate was, the benefits of this approach for your students, because obviously, you’re garnering insights from your students that are helping inform your efforts. So just to again, give this sort of its due diligence and focus here, how is this approach benefiting your students?

Chad Lystad:
It benefits our students and it benefits our student leaders. And again, I’ll go into detail about what exactly we do here in a moment, but the way it benefits our students, the ones that are experiencing the programming is they are able to, number one, again, in real time, reflect on what it is that they just did, learned, experienced, whatever it is, whatever programming they just went through. They get to take a beat, think about what it is that they experienced and reflect out, which I think will help them actually remember the learning that happened. Also, the second portion of what we do is, we ask them what questions they still have, and then we flip that around to the student leaders to say, in real time, these are the questions that your students still have so that we can adjust and change course.
If 70% of students still have this remaining question, then we know what to talk about when we get them in the room again. It helps our students because it’s more customizable, in the way that we do our programming and in terms of helping the student leaders, they get that feedback in real time to know that what they’re doing is making a difference.

Dustin Ramsdell:
And I guess, this is just how my brain keeps contrasting this, but yeah, it feels like when you have those huge biannual surveys, it’s like usually they’re serving so many stakeholders. It’s a big knot to untangle, if you’re trying to make adjustments or are we sharing out this information? It’s a lot of data. We’ve got to probably sort it out and filter it and do whatever else. But that idea of like, well, we sent this quick survey to your students of your program, it’s super easy to just give that to you. It’s all relevant versus like, oh, there’s half of this isn’t even pertinent to you, so we got to figure out how to share that. So yeah, I mean, I think it’s going to be that way, where I think that’s what a student is going to crave or student leader is going to crave to be like, I’m doing this in the trenches.
I need to know what I’m doing well, what should be doing more or less of? And that it creates those sort of reflective learning opportunity. They’re in a position where they’re, I think needing that even, I guess, I want to say more so than others, but I guess just as much as anybody else, but they’re really trying to develop their skills as they’re a student leader. So, I feel we’ve covered broadly what this is why it’s important. So, I want to make sure that we give a brief moment to just make sure that we cover the how. So how are you distributing this? How are you gathering the insights? And how are you sharing them with the student leaders? So what’s the bare bones steps 1, 2, 3, I guess, of how you’re getting this out, gathering the feedback and then distributing the insights?

Chad Lystad:
And that I think, is the beauty of this is because it’s really, really generalizable. So folks in Res Life can do this, career can do this, you can do this if you’re a faculty member teaching a class. The way we did things was in an orientation scenario. So I’ll talk specifically about that. But I think for you and your audience, Dustin, to take a step back and think about how they can, whatever programming they’re doing, they can use this approach. So in our orientation, what we do is, a number of programming beats. We’re trying to teach them this thing. Is it, what advising looks like? Is it, how do you keep yourself and your community safe? Is it, a programming around community service? So, we had them do these big things, and I’ll use our dramatic dialogues as an example. So at Concordia, the students all see these series of vignettes where actors portray some challenging situations around bias-related incidents or sexual assault or alcohol, and it’s these vignettes that they watch and then they reflect on, wow, those are pretty problematic behaviors.
And then afterwards, they do just a quick debrief. So here’s where the assessment piece comes in. So what we do is we have our student leaders just hold up a QR code, they scan it, and importantly here, I only ask two questions. Number one is, in this example, what are you going to do to keep yourself and your community safe? Question number two is, what questions do you still have? So I’m at my computer while all of these things are happening in various classrooms around campus, QR code gets scanned into Qualtrics, and then I can see all these results come in, in real time. It takes three minutes, maybe five tops. So I get all these results, I’ve got them all in a spreadsheet, and what I can do is… And I keep saying in real time, it’s not necessarily in real time. It might take me 10 to 15 minutes to do this, but what I can do is, I can compile then all of their responses, and I separate those two questions out.
So, I separate out the question about learning that happened, in my example, it’s keeping yourself and your community safe, but it could also be what roommate agreements have you come to within Res Life? What internship hopes do you have in career? Right? I separate the learning question out, and what questions do you still have, question out. I send the, what questions do you still have, to the student leaders who are working within the orientation clubs. I also include stakeholders. So I’ll include our Vice President, I’ll include people in various academic departments. Sometimes, I’ll include the President of the college if I really want the president to see what’s going on. So the student leaders then have access to, what questions do you still have? And they can sort and filter to, oh, here’s my group of students. Wow, they still have questions about this.
I can change the way I lead my group now, because I know where their heads are at. The learning question, what I typically will do is, I will compile all those, and then honestly, I’ll put the entire database into a ChatGPT and say, categorize these and give me one verbatim example per category. And then what I can do is, send out that to the interested stakeholders as well. So, it’s a really fast way of doing this. It’s a really, I think, elegant and simple way of catching, what did they talk about? What did they take away from what they just experienced? And what do we need to do moving forward to address the questions that they still have?

Dustin Ramsdell:
And just to make it clear… I feel like it’s impressive that it’s quick in the gathering and quick in the analysis. I think sometimes there’s efficiencies. I would say more likely on the front end. A lot of folks would be like, oh, we have a QR code. People fill out, whatever else. But then, the analysis is where people maybe get kind of stuck up. And certainly, I mean, I don’t have the skill set to manually sort of discern necessarily, categories and all that in an efficient fashion. It would take me a while. So I think one, using a tool in the gathering so that maybe it’s just formatted or presented where it can give you maybe some basic summary insights and then leveraging something like ChatGPT or any of those other similar tools. They are really good at just taking some raw data and just gathering it together.
And yeah. I think, that’s really what’s important. I think, it’s just like you said, I don’t know if there’s a magic number, but I would have to imagine, especially, it could be if it’s quantitative, just give this rating or something. You could maybe get up to five questions, but I’m like, it’s got to be five or less, especially if you’re asking for comments. But I think that idea of two substantive comment questions, and then it’s like if anything else, it could be like we went out to an event or program, like, okay, yeah, we just want a liker scale. Like, hey, how much did you enjoy this event? Or your enjoyment of this event, whatever. Ask the question but…
And just give it a rating out of five. But, yeah, I think it is just a good practice. I think, when you’re just trying to distill down what is the most important thing that we want to know here, and what’s the simplest way to ask it, you’re just really simplifying it. You’re not… Because I think, it is that idea that’s where you kind of balloon out to, well, I mean, honestly, yeah, there could be 20 questions that we ask about this one orientation session about every single little piece of it, but just organically getting, what did you learn and what do you want to know more about? You’re not preempting that, with the way that you’re phrasing the question or whatever else.
And you gave your personal example of how you were using it. But yeah, there’s no shortage of ways that this could be applied across the student experience, whether it is, yeah, career services, residence life, student activities, orientation, academic advising. And I think, and this may be kind of where you go later, so I won’t steer there too much, but I think, just hearing that you are using two great tools to help with the facilitation of this, I think is great food for thought as folks are sort of processing this. So, I’ll leave the door open if you want to further elaborate on our final question of advice and resources and stuff. But as we start to wind down, what do you see on the horizon for this work? So I don’t know how necessarily, you’re giving kind of the orientation example, how widespread the utilization of this is currently, if it is going to expand or anything else? But I guess, yeah, what’s on the horizon for this pulse check assessment work at your institution?

Chad Lystad:
At Concordia College. Excuse me. Let me retake that. At Concordia, I see this expanding into other aspects of our division. So I see this expanding to student engagement, student activities. I see this expanding to various places throughout the college. I gave the example of, while I was working in orientation, I’ve moved over to student engagement now. So, I see this being implemented within student government. I see it implemented within various other programming that we do for student engagement. I don’t think it always necessarily needs to be a QR code. It’s my default. I like to go back to it, just because it is so quick and easy. But I think there’s lots of different ways there, that we can do assessment to get a pulse check, as students are experiencing whatever programming they’re doing.
I just think it’s important to remember, that it needs to be very brief and it needs to just be reflective of what they’re experiencing at the moment. You’ve mentioned a couple of times, going off of the, year in review assessment or survey, which is great. That’s wonderful. But I think, the idea that they need to do a quick reflection in the moment is important.

Dustin Ramsdell:
Yeah. It’s just going to be, I think, just more genuine and authentic, because I think otherwise, when you’re doing sort of the end of year sort of thing, you might get good insights, but I think they’re going to be just generic or bland or there’s just going to be stuff that I think, will be affirming or critical in just a very general and broad sense versus really precise. Because I could imagine, especially if you’re doing event management, pulse check surveys, that idea of wanting ask, what did you like about this event? What could this event have done better?
You might get stuff where it’s like the check-in table was in an awkward position, and it’s that idea of like, yeah, cool. Yeah, we’ll move that. A lot of people said that, and we easily could have made that adjustment. We just put it somewhere kind of arbitrarily. But that idea of, that is quick, genuine, authentic, relevant insights versus just, you hear about how could we improve events in the future from a year-end survey. It’s just, I don’t know, something general where it’s like it felt disorganized or something. And it’s like, well, is that the table? Is that the check-in process? Is that… It’s hard to maybe pinpoint exactly, what did they mean when they said these events were disorganized or something. So yeah, I think that authenticity and specificity is going to be what’s really, really helpful for those kind of things.

Chad Lystad:
And honestly, to further your point too, Dustin, I think for a lot of those things we’re talking about there is, satisfaction surveys almost like. What did you think about this? What did you like about this? And I think the beauty of doing a quick and real-time assessment is you can really get to student learning as opposed to student satisfaction, which student satisfaction is very important. It’s really important to know what students like and what they don’t like and how we can improve. But if we really want to get to, what did they actually learn from this experience? Doing that real-time reflection is important. So you go back to my example of, how do you keep yourself and your community safe? The responses that I got for that, again, remember these just-witnessed actors portrayals of really challenging situations. The responses I got from that were, things like always ask for consent, always, or I didn’t realize that I have privilege in some of these areas. So, you’re really getting a rich assessment of what is actually going through their minds at the moment after having experienced that.

Dustin Ramsdell:
And I appreciate that delineation because I think, I’ve experienced sometimes, and it’s like as I’m saying this, it feels silly, but it’s like you can kind of get where people are coming from, but there is an assumption of one or the other. Sometimes it’s like, oh, well, if they were satisfied, they must’ve learned something. But then, it’s like you couldn’t really even say necessarily what, but if you’re like, what did you learn? It’s like, oh, I learned all this stuff about consent and bystander intervention. But then, you might presume that they enjoyed it. It could be like, oh, well, it was really emotional. It was really tough. It was maybe at the wrong time of day or something. So you wouldn’t want to necessarily assume, oh, because you learned something and I know what you learned, you had a great time in the session. It was amazing and everything was perfect and no notes.
So, you’d want to either, just level set those expectations or just be super specific, I would imagine, with what you’re asking and what you’re trying to get. So yeah, I do appreciate that clarification, that student satisfaction, that’s always where my brain goes, because in previous roles and stuff, we were trying to get at that and sort of a student coaching environment where we always try to figure out, hey, was this call satisfying to you and was it helpful, whatever else, and things like that. But I think that is, a very just important conversation to have is how do we want to leverage this? What goal are we trying to achieve? And that’ll guide, what questions are we asking? How are we asking them?
So I think then, that sets up, wrap up here perfectly, just advice. But if you have advice or any other piece of advice or resources that you’d want to share on this, I think… It is sometimes you start introducing an idea and if somebody starts going on, it’s like, oh, we went way too far in the student satisfaction piece. We totally neglected the learning piece. So you already gave an amazing piece of advice, but anything else, advice or resources to share on this topic?

Chad Lystad:
Yeah, I’ve got a couple pieces of advice, and I’m going to be kind of repeating myself honestly, but don’t overthink it. I think that’s the biggest part of it. I mean, certainly put thought into what you’re trying to do. Obviously, you’re trying to reach… You set your learning outcome at the start and decide what’s the best way to get to it. But in terms of the assessment piece, I think, just being really careful about, what one question do you have? Or what one question do you have, that will get to, did the students learn what you hope they learned? It doesn’t need to be much more complicated than that. Certainly I think, another piece of advice is to bring other people into it. So that’s one of the mistakes that I made when I started doing this is, I was doing all this stuff just out of my own brain and I wasn’t hitting the mark many times. So in consultation with others say, hey, I want students to learn this. This is how I want to teach them the thing. What question do you think I should come up with?
So, lean on your colleagues. Honestly, I think in some cases, asking students for their feedback, on what they think they should learn and how best they should learn it. That’s another piece that we’ve really started to implement here at Concordia. That is, I think, an important piece of it too. And just try to keep it simple. The best question that we’ve started to ask folks… Not started, we’ve been asking folks, is that second question is, what questions do you still have?
It’s really, really interesting, because the first question will often have primed that. So if the question that we ask is related to community service, what questions do you still have? Many of them will say, what opportunities do I still have for more community service after I’ve done this? So being aware that the first question will often prime the second question is, I think, interesting. In terms of resources. Dustin, I really will just plug the institute for the curricular approach. I’ve been a couple of times. It’s tremendously helpful ,if you or your audience can find a way to utilize ICA as a resource, you and they would get a lot out of it. There’s some really great information there, some really great people doing some really strong work,

Dustin Ramsdell:
Because I think, we’re kind of trying to get morsels of the magic of ICA through these conversations and everything. But I think that idea of, if you… Absolutely, like you’re saying, don’t ever think it, start somewhere, start simple, and certainly you’ll probably iterate as you’re getting more feedback, more often. But if you start somewhere now, by the time ICA rolls around again, you might be in a place where it’s like, yeah, I want to workshop ideas that I’m thinking about or things that I’m doing. And it does seem like from, I’ve never been maybe I will this year. Who knows? But just seeing and hearing what does happen there from afar, that it does feel like it is such a great place for this innovation lab, this idea that it’s like all these people working on this same sort of, obviously the curricular approach, this approach to doing this work, and because I think that sort of niche and getting all those people in the same room, it just generates a lot of great insights and conversations, and dialogues, and stuff.
I think otherwise it’s like, oh, we’re casting a very big tent for this education conference. Everybody’s here, and you’re just getting super broad. Or you have to try to really work harder to find exactly what you’re looking for. But yeah, I mean, I think again, this being sort of what feels like a very important, useful foundational aspect of curricular approaches, to complement some of the other more larger longitudinal efforts that often happen here. Really appreciate you highlighting this work and making yourself available for this conversation. And yeah, just really great stuff. So, thanks so much for your time.

Chad Lystad:
Hey, thanks, Destin. It was really fun. I really, really appreciate having this conversation.