| |
About
News
Activities
Products
Links
Search
Contact Us
R2D2 Center
CHS Home
UWM Home
|
|
Technical Report - Qualitative Research Discussions, April
29-30, 2003 (Version 1.0)
Frances Harris, PhD
Conference Participants:
Frances Harris, PhD
Bonnie Kennedy, PhD, OTR
Noralyn Jacques, PhD, OTR
Kathy Rust, MS, OTR
Marcia Scherer, PhD
Pimjai Sudsawad, ScD, OTR
Mary Taugher, PhD, OTR
Introduction
The Qualitative Research Conference (QRC) was organized with the goal of exploring ways in which qualitative research,
as it has been developed in the general field of disability and, specifically, assistive technology (AT), could be made
relevant to outcomes researchers. It also was an opportunity to introduce other less well-known qualitative and social
science constructs to the disabilities research arena. We were particularly interested in examining and developing a
qualitative research model(s) that could be incorporated within an AT outcomes measurement system.
Needs Assessment
Outcomes research in Assistive Technology is presently in its most formative period. Efforts to develop classificatory
schemes, concepts, methods, and outcome instruments currently dominate AT outcomes research and practice. At present, ATOMS,
one of two national consortium projects devoted to developing these objectives, is working to create an outcomes measurement
system. The aim of this system, among other things, is to provide an interactive, critically structured database. It would
be accessible to consumers, clinicians, policy makers and all those who have a vested interest in both directing and enhancing
AT use.
As regard its general methodology, outcomes research is quantitative in nature. That is, research questions and results are
tabulated and/or manipulated so that they may be expressed in a numerical format. However, classificatory schemes, which form
the basis of those concepts underlying outcomes methodologies and instruments, contain numerous assumptions, attitudes, values,
goals, etc. about disability and technology. The effect is that seemingly objective numerical data are quite subjective in
reality and practice. Indeed, the results of outcomes research rest on the assumed integrity and utility of its
classifications and categories. Outcomes results necessarily presuppose that the aggregate data produced correctly represent
the individuals' experiences on which they were originally based.
An example from the anthropological literature: Schiller (1992) examined the seemingly neutral epidemiological categories used
by NIH and the CDC to identify and communicate with AIDS risk groups, (i.e., Hispanic or Latino communities, IV drug abusers,
the gay community). Through qualitative methodology, it was demonstrated that these categories did not resonate with target
populations. For example: just because a man has sex with another man does not mean that he will automatically identify
himself as "gay." IV drug users may not necessarily see themselves as having a drug problem. And Hispanic and Latino
populations are not a single culture; they encompass diverse nationalities and cultures. One result was that public health
messages not only were less effective with the target populations, they actually overlooked the fastest growing segment of the
population to become HIV positive - heterosexual women.
In this instance, the crucial issue is that categories that appear commonsensical and authoritative must themselves be
investigated as ways of perceiving and organizing the world. Diagnostic procedures and questions may reflect more about how
medicine, a branch of scientific research, organizes its own knowledge practices and less about how the phenomena they are
interested in understanding actually occur. A good outcomes measurement system necessarily depends on its capacity to gather
information in order to accurately reflect the complexities of the behaviors and activities under research.
These issues profoundly affect outcomes researchers interested in understanding the impact of AT use on people with
disabilities in ways that society as a whole considers valuable. Consider that the recently modified WHO ICF nomenclature
related to disability serves as a guide for outcomes researchers who wish to calculate the effects of AT on its users. The
ICF construct of 'participation,' an increase of which is generally regarded as an obvious outcome goal, contains many
assumptions about the nature and desirability of societal, work, and familial relationships that may or may not be realistic,
desirable, or applicable across all disability types and contexts. These categories and classifications need to be
investigated to discover in what contexts "participation" reflects a meaningful goal. The various meanings and connotations
of participation need to be uncovered so that researchers can develop instruments that accurately assess its multiple
complexities and uses.
In short, qualitative methodologies may prove a powerful strategy for exploring this, as yet, minimally charted terrain.
Qualitative research can deepen our understanding and awareness of how outcomes categories are constructed and the multiple,
often confounding, meanings they may carry. Qualitative research also provides additional information about how categories
are actually "enacted." That is, the complex ways in which the subjects of research (for example, an AT user's participation
or wheelchair) may act out/create multiple identities (i.e., AT user as patient, client, consumer, parent, employee) or uses
(wheelchair as mobility, as an extension of one's body, the different spaces they inhabit or activities they facilitate).
People may use their various assistive devices in ways unanticipated by the clinician, vendor, or engineer who designed them.
Without well-integrated qualitative research methodologies, the complementary quantitative research data may be inaccurate or,
worse, undermine the research question and mislead those depending on the data.
The Conference
Structure
Frances Harris set the agenda for and chaired the conference. The goal for the first day was to engage people in informal
discussions and responses to general themes around outcomes research and qualitative methodologies, essentially a survey of
concepts, terms, and assumptions. During the second day we were to focus on formalizing ideas for future projects involving
qualitative research.
Day 1: April 29, 2003
Initial discussions addressed the differences between qualitative and quantitative research and how to meaningfully integrate
these differences within a particular research project. Since outcomes research is traditionally understood to be a
quantitative enterprise, it was essential to gather views on how different qualitative methodologies and theoretical
orientations could be meshed with outcomes research goals. All conference attendees, while acknowledging differences in the
types of data gathered and methodologies, agreed on the essential compatibility and potential usefulness of qualitative systems
to AT outcomes research.
Outcomes variables identified by the ATOMS team such as cost, satisfaction, function, use/non-use, quality of life, and
participation were discussed. A qualitative study that examined the use of canes by the elderly and which demonstrated the
range of meanings these assistive devices held for their users was cited as an important example of the kind of contribution
qualitative research can make. Meaning, defined here as the range of significance a variable may hold, was noted to be an
important contribution of qualitative methodology to AT outcomes research. Because assistive technologies may hold multiple,
even contradictory, meanings for their users, these meanings may directly impact researchers' understandings and attempts to
measure them.
Also, it was noted that outcome variables are not discrete, but "bleed into one another" and are interdependent. That is,
'use' is directly related to how well a particular assistive device 'functions'. We see this often in the abandonment process.
'Function' and 'use,' in turn, impinge on how we understand satisfaction and participation. How we look at variables, how we
carve them up and connect them to one another was identified as a key area in which qualitative research can make a contribution.
Three qualitative methodologies used within the general field of disability and assistive technology in particular were
discussed: focus groups, interviews, and participant observation. Two essential points were noted in this discussion.
First, focus groups and interviews report information from the subject's point of view. In contrast, participant-observation,
as a method, is concerned with both observing behaviors, as well as performing unstructured interviews. That is, what people
say they do or feel, how they perceive their actions are better understood and interpreted in conjunction with an observer's
perception of actions. Observation provides both a context and a different point of view for people's perceptions of their
actions. It was concluded that a qualitative strategy such as participant/observation coupled with self-report measures,
including interviews and focus groups, provides a strong methodology for qualitative research especially as it pertains to
outcomes research.
Various strengths of different qualitative strategies were also noted. For example, focus groups may produce unexpected data.
Such data is often useful for creating first generation ideas. One may extract rich information from these data and use it to
form initial questions and more sophisticated quantitative queries. From that stage one can develop questions that can be
graded along, for example, a Likert scale in order to allow verification of qualitative data.
Another qualitative methodology that was discussed as being potentially useful to outcomes researchers is derived from "actant
network theory" (ANT), a product of science and technology studies, e.g., Callon (1997), Callon & Latour (1981), Latour (1987,
1996). ANT assists us in considering a set of entities (people, machines, ideas, relationships, etc) as they connect more or
less directly to the focus of our investigation - in this case assistive technology. It treats all these entities as "actants"
as they act and are acted upon in a network of practices. They define each other by their relationship to each other. This
actant network represents all the things that construct our subject of investigations. It alerts us to the pre-programmed or
prejudged responses and similarly loaded questions we are asking. The model begins to show us not only those actants
influencing the subject but the dynamic relationship among these actants. Feedback from one set of actions potentially
reconfigures the relationships. Classically, we see this at a micro level when students provide the answer they believe the
teacher wants. The teacher, in turn, becomes more responsive to those students, altering their predisposition to answer in
specifically defined ways. This, in turn, alters the approaches to learning of other classmates.
ANT is a powerful qualitative methodology to emerge from a multidisciplinary social science approach to technology and health
issues. It has a pragmatic applicability to outcomes researchers' needs in that it considers how previously homogeneous
categories may contain many voices, interrelated variables, and hidden discourses with far-ranging effects. It lays a
transparency over quantitative data exposing the historical depth and complexities that inform its categories.
An important point to note is the quantity of data that qualitative methods can generate. Depending on the specific strategies
selected and applied, qualitative research typically generates large amounts of data in the form of verbatim notes, transcribed
recordings of interviews, focus groups, videotapes, jotted notes, detailed "field notes" of observational research, and/or a
researcher's reflections in the form of a diary made during the research process. A single interview, for example, can produce
30 to 40 single-spaced pages of raw data. It is not uncommon for a single project to generate thousands of pages of data.
Mentioned earlier, a strength and challenge of qualitative research is the rich and extensive data that is generated through
it. A realistic tactical and resource analysis of qualitative research data typically begins during the data collection
process itself. Continuous review of data allows questions to be refined iteratively, and may suggest new lines of inquiry.
This is appropriate to qualitative research goals, which are typically interpretive or exploratory in scope (rather than
statistically predictive or descriptive), and, therefore, are open to modification and development as data is collected in
both the structured and organic sense.
Once data collection is complete, the researcher begins to identify patterns, themes, and concepts from the variety of
accumulated texts. This is done by examining research questions, both in terms of the aims and objectives of the project, as
well as the issues raised by subjects themselves. There are a variety of tools (methodological and software) from which the
researcher may choose. Depending on the goals of the project, the researcher will categorize/map/index and/or chart the data.
These processes lead the researcher who will begin to interpret data and provide explanations by finding associations between
themes and creating typologies.
Thursday, May 1, 2003
Discussions during the second day of the conference concentrated on developing a research agenda that would augment ATOMS
research goals. Specifically, three different kinds of projects were suggested.
1. Phenomenological interviews using the three-interview method (cf. Seideman, 1998). It was suggested that a number of AT
users be interviewed using this particular methodology in an effort to uncover the range of meanings and uses assistive
devices hold in the context of their disability.
Phenomenology is a 20th-century philosophical movement that has had wide theoretical applications within the social sciences
(e.g., Heidegger, 1962). In short, a phenomenological research strategy is concerned with uncovering an individual's
subjective experience of their world. Through multiple and lengthy interviews, an informant's experience of their "world"
(in this instance the "world" as a disabled person) is explored.
The purpose of using the three interview method (Seideman, 1998) is to: first, generate data regarding a subject's life
history; second, to obtain life histories in detail with respect to their experiences as a person living with a disability,
and third, to allow subjects to reflect upon the meanings of their experiences. In this case, the meanings uncovered by the
users of assistive technology are rendered in terms that are extremely individual and personal. The contribution of such
interviews in terms of outcomes research is twofold: to uncover heretofore undiscovered themes related to the use/non-use of
AT, and to broaden/deepen our understanding of already existing categories.
2. A participant/observation research project based on actant network theory (ANT) was proposed. Such a project in
assistive technology would choose several locations in which assistive technology is prescribed (such as a clinic, hospital,
and school). Through participant observation and interviews the researcher would (for a minimum of 2-3 months) observe
client-clinician interventions, interview clients and their families and/or caretakers, clinicians and administrators,
vendors, insurance providers. Goals would include examining assistive devices historically, observing and comparing clinician
goals and activities with those of vendors, designers, engineers, insurance providers, and social policy makers.
3. A third proposal was to run a series of focus groups for multiple stakeholders in assistive technology, including patients,
clinicians, vendors and insurance agents. Goals would include bringing to the foreground differences in goals and
understanding (for example, of responsibility for AT provision and use). The resulting data could be used in one of two
ways: first, as raw data from which to distill key constructs and meanings which, in turn, could be used to build research
instruments. Second, data could augment other qualitative research data to provide both depth and critical feedback and
structure to other qualitative strategies such as participant observation.
Next Steps
This conference considered the general contributions of qualitative research methodologies and their overall applicability to
AT outcome system goals. The next activity is to formulate specifically how qualitative data may be analyzed in concert with
outcomes research goals. This involves a multi-part process. In order to make this realistically manageable, the first four
parts of the first phase are identified, below.
- Define a concretized and formalized understanding of how qualitative research in general is analyzed.
- Integrate the formalized understanding of analytical process and program with quantitative methodologies/findings in
service to AT outcomes research.
- Provide a specific context in which qualitative research could be applied to a current AT outcomes activity.
- Determine and apply appropriate resources to data gathering and analytic activities to allow the quantitative and
qualitative programs to be complementary and fully productive as a partnership.
A small team, possibly two or three individuals could rapidly draft responses to the definition of an analytical program.
This would satisfy points one and two, above. QRC participants could then review the draft for comment. The intent of the
review would be to correct or amplify any issues left unclear or absent from the draft statement and to construct a response
to point three. The draft would then act as a framework to develop a general structure to respond to point four.
Summary
Qualitative research methodologies can provide several levels of novel and complementary utility for AT outcomes researchers.
The iterative, organic, but formalized program of data collection and analysis can clarify, create, and refine, existing
categories and concepts and their relationships among each other for organizations, individuals, technologies, economies, and
connections. They also can provide information that illuminates assumptions in existing quantitative instruments constructs.
The innovative outcome of this complementary approach can produce better quantitative information and a context through which a
broad audience of researchers, end-users, and policy analysts and decision-makers can access and comprehend the power and
expertise of outcomes work.
References
Callon, M. (1997). Representing Nature. Representing Culture, Paris: CSI, Ecole Nationale Superieures des Mines.
Callon, M. and Bruno L. (1981). Unscrewing the big Leviathan: how actors macrostructure reality and how sociologists help
them to do so. In K.D. Knorr-Cetina and A.V. Cicourel (eds), Advances in social theory and methodology: Toward an integration
of micro- and macro-sociologies (pp. 277-303). Boston, Mass.:Routledge and Kegan Paul.
Heidegger, M. (1962). Being and Time. (J. Macquarrie & E. Robinson, Trans.). New York: Harper.
Latour, B. (1987) Science in Action: How to Follow Scientists and Engineers Through Society. Milton Keynes: Open University
Press.
Latour, B. (1996), Aramis, or the Love of Technology. Cambridge, Mass: MIT Press.
Schiller, N.G. (1992). What's wrong with this picture? The hegemonic construction of culture in AIDS research in the United
States. Medical Anthropology Quartery, 6(3), 237-254.
Seidman, I. (1998). Interviewing as qualitative research. New York: Teacher's College Press.
The ATOMS Project and this work are
supported in part by the National Institute
on Disability and Rehabilitation Research
(NIDRR), grant number H133A010403.
The opinions contained in this publication
are those of the grantee and do not necessarily
reflect those of the NIDRR and the U.S.
Department of Education.
© 2004 - R2D2 Center at the University of Wisconsin-Milwaukee
Copy and distribute
freely, but use in its entirety and do not alter.
|
