It is mid-March 2020, in a week that has seen most institutions in Canada, the US and much of the world change how they teach, how they communicate and how they gather. Institutions are announcing the cancelling of in-person classes, the moving of classes to remote and online delivery, the closure or restriction of access to campus, cancelling or making optional course evaluations, sending staff home to work remotely as kids are also home from school,
Questions of access equity, including internet access and how to assess fairly in the midst of work and home disruptions in so many of our lives. Self are self-isolating to protect family members, others have returned from travel and self-isolate to protect their community. Kids in many jurisdictions are home from school and non-essential travel is curtailed, community events, concerts, sports (including NBA, NHL, Football season in Italy) and more are closed, paused or moved online. The Berliner Philharmoniker concerts are now only digital. Coffee meetups with friends are now FaceTime’s, Zoom meetings and Google Hangouts.
Some institutions are already cancelling spring and summer courses, and some have already cancelled current classes, though most are trying to move remotely. Professional organizations are approving 75% hour completions as completions for practicums.
Work, Higher education, and life are disrupted, reprioritized.
And in the midst of it, we are seeing each other as human. As family members with partners, kids, cats and dogs. As team members in figuring out our options. As people frustrated and grieving lost opportunities and moments, and some grieving those we have lost to the virus. A massive rewriting of daily life, and of the roles and ways we interact.
How we teach, how we learn, how we assess…what does a good early ending look like when we don’t have the usual milestones? What does mentoring look like when we can no longer hand over a book? What is fair for the students? What is credible and will look reasonable later to our peers and to anyone reviewing this year.
We will redefine what it means. Each time we chose what makes sense, what is honest, what makes sense, what is needed. Each time we see the human in the learner, in the peer and in the educator. We are all in extraordinary times, let us be extraordinarily considerate and remember what will matter most is what we learn about the world from each other’s choices. Let them be good ones.
For resources on teaching: keepteaching.ca
For peace of mind, remember this does feel and is new and yet the birds are still returning to their summer grounds.
In many projects, I find myself in a room of stakeholders discussing the data. As questions emerge, it becomes evident that they are interested in just-in-time data.
It takes skill as an evaluative leader to guide the discussion through increasingly complex datasets and even more complex interpersonal and power dynamics: being able to utilize and adapt pivot tables, charts, and conditional formatting can inform evidence-based change through engaging data discussions with stakeholders. This workshop is intended for individuals familiar with data entry in typical Excel datasheets with rows and columns.
Through this workshop, you will develop technical practice competencies related to analyzing and interpreting data (new CES competency 2.8) and group facilitation skills related to data discussions (new CES competency 4.5). Recognizing and planning for the use of pivot data tables and charts that auto-update has additional implications for effectively using human, financial and technical resources (new CES competency 4.3).
Session: CES Workshop, Halifax, May 29, 2019
Register via CES conference registration select member or non-member registration and then select Plotting a Dynamic Journey May 29th workshop list www.c2019evaluationcanada.ca
Interested and unable to attend CES, complete the contact me form to be notified of upcoming workshops or to arrange to host a workshop.
About Carolyn Hoessler:
I am passionate about respectful, meaningful, efficient discussions that unlock pathways towards meaningful change and alignment of goals, activities and measures. I bring to our conversation, 12 years experience in facilitating learning, assessment and program development within higher education and beyond. And as a methodological geek, I draw on quantitative, qualitative and mixed method analyses to explore questions of change, contribution, and evidencing value.
By the end of this full-day workshop, attendees will be able to:
- Create pivot tables and then adapt them to address additional questions
- Create charts and visuals that auto-update from refreshed pivot tables
- Organize data effectively to create pivot tables with consideration of row identifiers, demographics and the aggregation across rows but not related columns.
o Note: if you have 3 scorers rating the same outcome variable represented as 3 columns, the pivot table will treat each column as a separate variable (and separate column or table).
- Conditionally format data and pivot tables based on both colour scales and specified criteria
- Thematically code qualitative data and organize results with a pivot tables to analyze and summarize for both simple coding and multiple codes per quote.
Please note that this session will include demonstrations in a Mac interface. There will also be Windows-based examples and instructions. Hands-on guidance for Windows-based will be provided. To request a Windows-based workshop, complete the contact me form
Subject Matter Expertise: I have facilitated national and international workshops on statistics from a practitioner lens, mentored faculty on qualitative and quantitative analysis in the evaluation of course and program design for over 6 years, and taught graduate courses in statistics, qualitative research methods and mixed methods, I routinely conduct surveys and focus groups with responses from up to 200 students and analyze them in excel for internal reports for program improvement, and work in large-scale datasets.
Facilitation Experience: I have led workshops on teaching and facilitating learning (10+ years), on statistics nationally (CES 2018, ISSOTL 2017, EDC 2011, STLHE 2009) and internationally (ICED 2010), data visualization nationally (STLHE 2018, EDC NJAW 2018) and on evaluation in educational development nationally. I have also taught small group sessions on excel for evaluation project teams.
Why it matters:
- 60% of students are experiencing high levels of anxiety (
Dr.Dreher citing Wolverton, 2019)
- Top reasons employers frequently fire millennials include low confidence in self and decisions, collaboration, overconfidence (a disconnect between what can do and what think they can get) and lack of accountability.
Dr. Dreher connects the high levels of anxiety to
What educators can do:
The goal is to build a strong internal locus of control a sense of connection between the decisions and actions we make and the outcomes that happen.
First, Let’s model it
Focus on the factors we can control as educators. While factors we cannot directly control as educators very much do impact our students, so too can what we teach and how we teach it. When challenges come up in the classroom, note the parts we can control. The slide projector stops working – we have laptops and the ability to email slides. The internet is down, we can shift the order of our activities.
Second, Let’s expect it
Speak of the what’s next, and ask what they can do. In our first class, I note: Life happens, first be safe then call or email me. I also ask what resources are available on campus as it helps the newer students learn from those who have been on campus for a while. Asking what part of the readings most relates to your project? What’s your next step on the project this week? Who could you do an informational interview with to learn more?
Third, Let’s reward it
One of the ways to change mindset is to change patterns behavior. Rewarding indicates which behaviors to increase. We shape behavior; constantly reinforcing what has been learned prior or disrupting it. I have my students write a scope proposal (what will look at, time estimates), then an update, then a draft, then and a final version with a note indicating what they changed. The assignments intentionally build so that early decisions and feedback inform later stages. Their decisions matter! How and if they respond to feedback matters! It also helps identify common issues early like too large a question, leaving the literature review to last minute. It has them identify their challenges in the updates and a revised timeline, and also the edits they made from the draft to the final version and the areas of the rubric they worked on so I know what to look for (Bonus: saves searching for what has changed!)
The goal for many is to be prepared for careers and get hired.
Yet in an era where top companies no longer require a degree and a degree is not a golden ticket, what does it means to be prepared?
Degrees are no longer exclusive indicators of success and prerequisites for job interviews. Rarely will a company ask to see your transcript and few will care if you graduated fifth or fiftieth. Even for graduate school, standardized tests and references carry much of the weight in addition to degrees.
Yet there are things that both job seekers and educators can do.
1) Stating naming what you can do and have learned: mine the outcomes of your program. Increasingly, programs have program outcomes as well as the course outcomes listed on syllabi. Consider what you have practiced, shown you can do, and gotten better at from your summer jobs, hobbies, volunteer work, and family contributions.
2) Create networks – Connect with classmates, faculty, speakers, parents, and friends. Find out more about the many ways you can work in this world, and learn what a day looks like then try it out as a volunteer or part-time job.
3) Talk about your skills in context: connect experiences to what you can do and what makes you a good fit.
Universities and colleges:
- Show examples of the skills in context. Even writing a paper is about persuasion, communication, synthesis, and research. It is about learning the language of a group and working from there.
- Embed career readiness into courses and programs including training in principles of teamwork, strengths, and conflict resolution.
- Show examples of the skills in context.
- Create and share meaningful relevant program outcomes that identify for students what learning they have gained.
- Assess and recognize competencies through competency-based education
- Provide the scaffold and expectation for developing a portfolio where students can collect, name, and showcase evidence of their learning.
Finally, it is time to end the debate of education for career or life or society. Communication, interpersonal skills, creativity, and teamwork matter just as much on the soccer field, the neighborhood fundraiser, in research teams, and on the job
They say that the one constant in life is change, and this academic year has definitely been about change.
I will be finishing at the University of Saskatchewan on January 31st. I have accepted a two-year position as a Curriculum Development Consultant at Ryerson University, in Toronto, starting in February.
In the fall, I also got to experience being a student again, and seeing the classroom experience “from the rows” after years of being “at the front” and working with instructors through consultations and classroom observations. The experiences add to the lenses through which I could see alignment, expectations, engagement and communication manifest in our classrooms.
Last Friday I also facilitated my last (for the foreseeable future) TA training session. It is where I started on this journey as an educational developer and there have been many such sessions throughout almost 12 years.
I wish you all the best in your journeys as well, which even when not easy may they be one of our greatest teachers.
(un)Guided Transitions: Why our students are unprepared
The complaint that high schools poorly prepare students for university rings frequently in the halls each fall, with students perhaps knowing the content and having gotten A’s seem unable to read a syllabus, to complete readings, take notes and engage in classrooms appropriately. Scoffed at in comparison is the equivalent complaint that our graduates enter workplaces perhaps knowing the terms and theories and having gotten at least B’s seem unable to attend meetings, prepare relevant documents, engage in workplace meetings and with clients appropriately.
What’s going on? The content knowledge and skills are there. Initiative and being “good”, however looks profoundly different as the expectations and norms shift.
To be a good student in high school, one must be within appropriate pace within the class working with and completing the tasks assigned by the (flexible perhaps) deadline, participation in class is to say relatively little and to be a team player among peers.
To succeed in university is more individualistic in many courses, to exceed in the pace of reading and learning, to speak when opportunity exists, to demonstrate one’s keenness within social norms that privilege quick and connection-laden answers. Even something seemingly tangible like expectations for good essays and sufficient citations are different.
To succeed in the workplace is to place group before self, yet leverage group interactions to show one’s capability as a team player. To paradoxically either shine to be mundane: To excel when in the appropriate seat or with supervisor-identified problems, and to blend in when seen as one of many workers. Unique edges or common middle that must be in sync. To seek to be unique in the middle is perceived as discordant and outside one’s role.
So the transition path from guided growth to individualistic intelligence to context-attuned shining mundaneness, speaks to an unspoken set of norms and cultures that our students are ill-informed and prepared for.
At the crux is that to develop to succeed at one game leads to ill-success at the next game. To be dutiful in waiting for the teacher succeeds in high school but fails in university. To predict where the teacher is going fails in high school but succeeds in university. To be able to work in teams but privilege individual growth succeeds in university but fails in many workplaces. To blend in allows for comfortable success in workplaces but risks potential lost opportunity in university, particularly where accessing graduate degrees or reference letters matter.
How do we already strive to scaffold this transition of expectations and norms? What do we already explain, and what remains hidden and without words? What strategies, examples, cases, rubrics, senior peer mentors, informational interviews and more do you use?
In educational development, evaluation, research coaching, and consulting, we face the challenge of describing the work we do, even though it seems deceptively simple while we are in the midst of bringing our knowledge and skills in order for others to achieve their dreams and mitigate concerns.
For years, I have wrestled with what do we call our work, and in a time of local and national discussion about our work, it is, for me, a time of deep reflection on how we position the work we do.
- Position: Do we see ourselves as offering from a place of expertise, position ourselves as equals and partners, or helping like a helper (guiding and carrying a load) for others to achieve?
- Goal: Are we focused on transformative change to their ways of being and doing, or simply helping them make another year and easing their troubles?
Throughout our work, we may advertise and prescribe one focus or position, or we might tailor our communication and cycle through the three positions and two foci throughout a professional relationship with the client, instructor, student, or researcher.
The final question is perhaps the most challenging: 3. Power: Who really gets to choose? Do we? or even more messily, do those we seek to work on, work with, and work for get to choose?
What concerns me about descriptions that speak of equal partners, is the third question, who is that decides that we are equal.
In our work, I can call declare that a researcher, colleague, faculty member, instructor, community leader, organization leader… is my equal, but only they can declare that I’m their equal. If I declare that we are equals, I am assuming a superior right and status from which to make this declaration.
To use an analogy: In the German language and culture, one must be granted permission to address someone with “Du”, the informal and personal version of “you”, for to do so is to declare that someone is my equal. I can only offer that someone else is my equal, and only they can declare that I’m their equal. To unilaterally declare (i.e. addressing someone with “du”) presumes that the person so declaring is unarguably the person of higher status (e.g., boss, grandfather).
Similar to the wise consideration required as we walk the tightrope of celebrating our work without claiming the work of others, we to need to be respectful when we declare our position and focus. Unilaterally labelling ourselves as another’s equal raises the concern that in doing so we are assuming a superior status, particularly when communicating to new contacts and starting work together. And when we label ourselves part of transformative change, we risk excluding those with ameliorative concerns or being seen as pushing for change faster or broader than what is being sought.
It is a struggle, no doubt about it, in trying to figure out where we are in the paradigms when we are flexible and each person we work with pulls us to where their perspective is. There is a risk that in providing what they want, they may lose out on what they need. The question becomes who chooses the focus and our position! I believe we can no more abdicate that choice than we can dictate it, and that makes things most challenging each time we describe what we do or sort out our role.
For all those in research, teams, educational development, university life, community organizations, consulting, and evaluating, I look forward to hearing the words you choose, the adaptive balances you strike!
Another day, another conversation about the missing nuanced view of stats and data. Despite the saying, statistics is not all lies but neither does it magically produce nuggets of Truth.
Stats is a mathematical box that takes what we give it and crunches the numbers to produce a single set of summary numbers (t(3) = 2.54, p < .05). What we sometimes forget, or never really knew, was that the numbers are related to our decision as researchers & interpretations as readers. The same dataset will always produce t(3) = 2.54, but p < .05 only occurs if we assume “one-tailed” but would get p > .05 if assuming “two-tailed” (handy calculator). The meaningfulness & usefulness of the difference requires yet another layer of interpretation and decision-making.
For a quick summary of what stats can do, can’t say, and should be questioned about:
The numbers produced by statistics are very tempting. They are consistently produced by a process created by smart people, they are mysterious and are given weighty titles of being empirical, statistically proven, and accurate. And when interpreted as being about what is specifically measured and specifically tested they are true.
The devil, or in this case, the truth is in the details. What exactly is being measured? Which analysis is used? What variables are or are not included in the analysis? How are they positioned, weighted, and ordered? What level of difference or relationship do we set as enough? So many choices must be made in good statistical analysis, and it is these choices that you can judge.
The leap to declaring “Truths” about life’s problems, other contexts and the future is indeed a leap of faith. We can make good decisions, bolster our arguments for why others need to believe us, and in the end there is simply trust and the judgement of reasonableness.
Raising key questions, yesterday’s published quote of HECQO recommended testing of “soft” / “essential” skills (math, communication, problem-solving, critical thinking, teamwork…) upon entry and graduation to PSE. (Read news article at www.thespec.com/news-story/6327523-young-grads-need-to-brush-up-on-3-r-s-employers-say/)
The question or measurement is an important issue in Canadian PSE. So I ask two questions of the individuals within the PSE community whom I greatly respect: What do we really want to know? What would it mean to get measurement right?
First, Why would we measure?
I can think of two reasons:
- Learners period. Regardless of who sets the goals and processes of their learning, students care, educators care, and society cares. We may see success differently, but generally there is a continuation of learning, of growth in knowing and seeing, and a contribution to society through their gained ability to wrestle with big (and small) issues and tough (and daily) problems.
- Contributing our understanding of teaching concerns and successes is an important role of measurement. Do students really know “less math” now than 10 years ago when they arrive? Is there a shift in language and conception that mismatches with expectations? For those who like numbers, quantitative measurements can provide a more comfortable ground for addressing and celebrating. For those who like narrative but talk with those who like numbers, conversations become more possible.
Second, What are we measuring?
- Knowledge or Expertise: I may know a formula or close reading, and still not be able to apply it to a public debate about water sovereignty or creation of a new innovation. Knowledge (especially at the recall or near-application level) that might be tested is not a substitute for assessing thinking process, reasoning & decision-making based on knowledge, and the nuanced application of knowledge to a complex situation that requires recognition of deep features rather than superficial ones.
- Specificity: Are we sufficiently measuring the relevant breadth of knowledge when we measure numeracy, literacy, writing…(informational literacy…statistical literacy…)?
- Building on the work of others: How might the VALUE rubrics of AACU be adapted? What are the limits of measurement and the tensions of measurements identified in the UK, Europe, Australia, USA? What can we learn from colleagues coast to coast to coast?
“Knowledge” like measuring foot size for running speed
Fundamentally I think the question of measuring “knowledge” is misnamed and in the misnaming there is a risk of mismeasurement. “Application” is insufficient too, as is “job-readiness” for I don’t think we could be prepared for every task or job all of us do. Knowledge is likely related and likely necessary but is not sufficient for what we are seeking for our learners.
Instead, I think we are seeking the foundational concept of expertise in its many forms. We want learners to go from superficial snap trial by error that leads to incomplete or inappropriate answers -> to seeing the real (deep) features of an issue, to thinking through the process, making decisions about what matters and what to do, and then creating and applying in nuanced ways the ideas, facts, processes and capabilities of their disciplines. Whether it is to HR decisions, communication rhetoric, bridge spans, server backups, policy creations or critiques, society needs more from our students and us than simply knowledge.