Intentional Conversations are one-on-one meetings between student staff and their residents guided by a suggested set of questions and prompts that are developmentally appropriate and situated within the context of a resident’s experience. This post is one of a multi-part series examining and providing suggestions for residence life and education departments that utilize Intentional Conversations as an educational strategy. It coincides with the release of new enhanced features in the Roompact software for implementing and tracking these conversations. Posts included in this series are:
- What are Intentional Conversations and Why Should You Use Them in Residential Education?
- How to Structure Intentional Conversations in a Residential Curriculum
- Developing an Intentional Conversation Curriculum Guide for Student Staff
- 100 Questions You Can Use for Intentional Conversations in the Residence Halls
- Don’t Be Creepy: Training Student Staff For Genuine Intentional Conversations
- How to Track and Assesses Intentional Conversations for a Residential Curriculum
The utility of Intentional Conversations is in part dependent on the ability to demonstrate student learning is occurring and revise and enhance the strategy based on assessment data and feedback. In the previous posts in this series, we examined Intentional Conversations as an educational strategy, discussed expectations and structures, outlined conversation guides and questions, and proposed how to conduct staff training. In the following post, we discuss how to track and assess these conversations.
The integration of data collection, analysis, and other assessment techniques is integral to the development of a residential curriculum and educational program. This includes the range from formative to summative assessments and from the individual student level to the broader programmatic level. Assessing the learning occurring during Intentional Conversations is an important component of any comprehensive assessment plan, and yet, given very individualized and contextual nature of Intentional Conversations, it can be difficult to achieve.
When evaluating Intentional Conversations, it is important to make a distinction between tracking their occurrence, and a resident’s satisfaction with the conversations, and assessing the student learning that is occurring during and between them. The former is a significantly easier endeavor than the latter, but both are important pieces to the overall assessment and evaluation puzzle.
Tracking Intentional Conversations and Student Satisfaction
Tracking Intentional Conversations is important to ensuring that the expectations placed on student staff members are being met and in ensuring that all residents are provided the opportunity to participate in the conversations. This tracking occurs with staff members (How many of the required Intentional Conversations have staff members completed during defined time frames?) and with residents (How many times has a student been met with (or not)?).
Another component of this tracking may be follow-up surveys to collect feedback from residents about their satisfaction with the interaction and any feedback about the Intentional Conversations program itself. It is important to note that this is different than attempting to assess the learning occurring during the Intentional Conversations. Although assessing learning may be brought into a follow up survey, much of it may rely on student self-report—not the most reliable strategy.
Working with Data from Intentional Conversations
To assess learning and the individual Intentional Conversations, many institutions rely on data collected through student staff member notes. This information, unlike the quantitative nature of tracking and gathering feedback on Intentional Conversations as a strategy, often relies on more qualitative measures. This realization is key if one is transitioning to a residential curriculum model. Just tracking the occurrence of Intentional Conversations is no better than counting heads for attendance at programs. What matters even more is a demonstration of student learning.
Individual Resident Learning
On an individual level, a student staff member should check for resident learning during the course of an Intentional Conversation as it is occurring. Methods for achieving this could be incorporated into student staff member training. Students could end the conversations with a check for understanding, asking residents what they reflected on or learned about themselves, or asking residents to state their goals at the conclusion of the conversation. Examples of summary questions include:
- What is one goal you are setting for yourself this semester?
- What is one take away you have form our conversation today?
- How will you know you’re successful in achieving [this outcome]?
Community Themes
After completing an Intentional Conversation, a student staff member often records notes about what was discussed. Consider developing a consistent set of keywords or tags that denote topics or goal areas that arise in these conversations. These tags could include topics like: homesickness, academic difficulty or success, developing multicultural competence, involvement, etc. With this data, a professional staff member can review staff member notes and codes, double check their work, and run frequencies and look for emergent themes. These themes can include common issues, struggles, or successes that a student may be experiencing. After checking frequencies and coding for themes, certain emergent connections can be made that will allow the professional staff member to write summaries and suggest implications or areas of growth that may be more common to a residential population.
Rubrics
Another evaluation strategy is the use of rubrics. Rubrics are tools that allow for the evaluation of student learning and development across a continuous scale. The example of a rubric below demonstrates how a student moves through various levels towards the achievement of desired outcomes. In this case, the rubric is measuring “Academic and Career Planning Skills” and “Studying, Time Management, and Academic Skills.”
If written appropriately, rubrics can provide very concrete evidence to watch out for when speaking with a student. At the end of a conversation, a staff member can place a student on this scale and check later for movement along the continuum. If a curriculum is effective, a student should be able to move through the successive stages over the course of their time in residence. This type of data can also be used to evaluate departmental learning outcomes to gauge if they are reasonable achievable by students, or if they are too ambitious or not ambitious enough.
Demonstrating Learning and Effectiveness
If you do follow-up surveys with your residents to assess student learning, you may consider doing a pre-test post-test type evaluation. With this model, you could ask residents to respond at the beginning of the year with goals or issues there are facing in the coming semester or year. At the end of the term, you could follow up with questions about their achievement of their stated goals, what their experiences were, and what they learned. You may also consider building this into the questions that student staff members ask.
Conclusion
Moving towards Intentional Conversations as an educational strategy has the potential benefit of enhancing student learning and allowing for more customized educational experiences. Tracking and assessing these conversations is an important part of this shift. This includes both the tracking of conversations, soliciting feedback on their execution, and looking for evidence of student learning. Going in with a plan will ensure that your Intentional Conversations will remain as their name implies, intentional.
Key Questions
- How are you tracking the occurrence of Intentional Conversations?
- What will you do with any data collected through student staff member notes?
- What technology are you using to help with these tracking and assessment efforts?
