Friday, July 8, 2011

Measuring the Unmeasurable: A Case for Narrative Inquiry in First-Year Research

Bryce Bunting
Bryce Bunting is the program manager for Brigham Young University’s Freshman Mentoring program, a learning community initiative that enrolls and provides peer mentoring support to all 6,000 of BYU’s entering freshmen students.  Bryce is also heavily involved in new student orientation programming and moonlights as an instructional designer.  He blogs about various issues pertaining to higher education, teaching & learning, and anything else he finds interesting at www.brycebunting.blogspot.com

In a recent posting  John raised some intriguing questions about measuring the impact of our work with first-year students (e.g. How do we measure whether or not students love their university experience?  How do we measure transformation?).  These are, no doubt, important questions for FYE professionals to ask.   And, if we see our work as more than just boosting retention stats, they are questions that we have to grapple with.
The trouble with questions about “love,” “transformation,” or “the impact of a mentor” is that these things are tough to measure.  Granted, researchers can operationalize concepts like love and develop instruments that measure proxy indicators that the more fuzzy concepts are present (e.g. we measure “engagement” by measuring a variety of indicators including time spent working outside of class, interactions with faculty members, study w/ peers, etc.), but this approach has its problems.  Before readers’ red-flags go up, this is not a rant against quantitative research or operationalism—this type of academic work has its place and helps us to answer critical questions about student experience.  However, while p-values, standard error estimates, and confidence intervals, are sometimes seen as the coin of the realm in academia, for some questions (i.e. those mentioned by John), particularly those with deep personal relevance for us, the traditional quantitative approach can leave us feeling unfulfilled, still questioning whether or not our work has really mattered in the lives of individual students. 
 The good news is that there are other research approaches coming to be recognized as just as viable as more traditional quantitative approaches.  One that may hold particular promise for FYE researchers is narrative inquiry.  A blog post is not the place to attempt an exhaustive explanation of this discipline; rather, my intent here is to provide a brief sketch outlining the general landscape of the field.  For those wanting more depth (and much more sophisticated) than is found here, these are some good places to start:
“Stories of Experience and Narrative Inquiry,” Educational Researcher, 19(5), 1990
Narrative Methods for the Human Sciences, Catherine Riessman, 2007
Narrative Inquiry, Jean Clandinin & Michael Connelly, 2004
Touching Eternity:  The Enduring Outcomes of Teaching, Tom Barone, 2001

Narrative inquiry is a sub-discipline within the broader field of qualitative research.  Based on the premise that storytelling is an inherent human quality and that we live “storied lives,” narrative researchers use narrative data sources (e.g. story-telling, journals, field notes, interviews) to study how we experience the world and then make meaning of these experiences.  As an example, John’s question about love for an institution could be addressed by interviewing graduating seniors, soliciting stories from sophomore students, or analyzing  journal writing from a freshman seminar course, then identifying meaningful themes that cut across the entire set of “data.”  Ultimately, the researcher would aim to use these narrative sources to construct a “meta-narrative” that tells an overarching story related to the initial research question. 
As with any research methodology, narrative inquiry has its critics.  Common criticisms of narrative inquiry include a lack of generalizability, reliability, and validity (similar to the arguments against most qualitative approaches) and these are all fair claims.  However, as a general rule, narrative inquirers don’t view these criteria as goals of their research.  Rather, their focus is on transferability, apparency, and verisimilitude.  To narrative researchers, the narrative approach is a way of addressing personal and human dimensions of experience in ways that quantitative data cannot—they are working from a completely different set of assumptions about research.  Jean Clandinin and Michael Connelly (arguably the most well-known and well-respected narrative researchers) put it best in their paper “Stories of experience and narrative inquiry” when they said that “stories stand between the general and the particular, mediating the generic demands of science with the personal, practical, concrete demands of living” (1990). 
Lest this post be viewed as a one-sided push for a problem-free research approach, it is important to recognize narrative’s dangers and pitfalls.  Because narrative asks researchers to construct stories from qualitative data, the possibility exists for individuals to fake the data and concoct pure fiction, based on nothing but unfounded assumptions and only anecdotal data.  Additionally, narrative could be used to misrepresent experience and promote deception.  Finally, some narrative researchers (particularly when stakes are high) may present only “Hollywood stories” where, for example, students love everything about their university experience, become deep learners, and are transformed by their college experience for the rest of their lives.  But, it bears mentioning, that similar claims could be made of virtually any approach to research, even quantitative methods.
While narrative inquiry may not have the tables and statistical significance that sways some political and campus leaders, there is rhetorical power in human stories that we can relate to and that elicit similar stories from our own experiences.  What’s more, narrative inquiry is gaining traction among educational researchers, as evidenced by the recently formed Narrative Research SIG within AERA, the convening of the first ever Narrative Inquiry conference this last May, and the well-respected work of Clandinin and Connelly.  Most importantly, narrative offers an approach for answering those questions, like John has posed, that are deeply meaningful to us personally, and whose answers we use to measure the impact of our careers.

Thursday, July 7, 2011

College as Lab for Real Life

John Gardner
President
When I look back on my own college experience and ask myself in what context did I learn the most that influenced how I practice my profession 4 decades later, my answer is: student government! I learned that my small, liberal arts college was a lab for real life. I learned about how organizations function; how decisions are made; how and why conflicts develop; how vested interests get pursued; what brings people together to do the right things. As I look back, particularly in contrast to what I just saw the British government doing during my visit to the UK in June, and now what my own government is doing—or not doing re the crisis around raising the US debt ceiling—I realize that I learned adult behavior.
And I am desperate to see some adult behavior in our Congress. I bet even some of our students are. This makes me wonder what we can teach our new students this fall as they join us about adult behavior. How are our colleges and universities models for adult behavior, unlike our Congress.
What will our students see the adults doing?
Will they see the adults putting the institution’s (country) ahead of more parochial interests?
Will they see examples of civil discourse?
Will they see examples of reasonable thought and action, based on evidence and facts as opposed to ideology?
Will they see ideology trumping all reason and pragmatism (e.g. we cannot absolutely and under no circumstances raise the organization’s revenue stream—we can only cut expenses—can any of our students manage their own personal budgets his way?)

Will they see any examples of real statesmanship?
Will they see examples of leaders saying “I won’t come to your meeting for discussion, or stay in your meeting, unless we refuse ahead of time to discuss certain topics!”

I learned a long time ago that there are many big things in life I do not and cannot control.

So I have to focus on what I do control. I can’t control what happens in Congress but if I were still on a campus I could influence, and even control in some ways, what my students would see us adults doing.

I so hope my colleagues on campus will welcome our students this fall to a bastion of civility and rational discourse; and to a community that is as generous as possible to all and lacks the mean spiritedness we see so prevalent in Congress.

What our students will see this fall is up to people like you who read this blog. Please let
them  see us acting like adults.