General Interview

From EvaluationWiki

Jump to: navigation, search

Contents

Time Commitment, Success Measures, At-a-glance Overview

Process Name: Interview Set-Up, Protocol Development, and Report Writing

How long will it take to complete? Depends on how many and the number of people

  • Pre Interview: 40 hours
  • One Interview: 30 minutes to 2 hours
  • Transcription: 4 minutes for every minute of the interview
  • Analysis: 40 hours

How will I know I’m successful?

  • Each interview is set up: time, place, participants
  • Participants show up
  • Interview protocol is developed and reviewed by all necessary stakeholders
  • Participants feel they can be open and honest
  • Protocol questions elicit the necessary information
  • Report is written in a timely manner
  • Report answers all questions addressed in the grant or the evaluation plan
  • Data in the report provides feedback for stakeholders on possible improvements or continuations of features
  • Client is satisfied with the quality and timeliness of the process

At a Glance

Calendar Items to Bring
  • Start-4 weeks: Establish needs, expectations, location if face-to-face, participants
  • Start-3 weeks: Ensure you have location and time
  • Start-3 weeks: Invite participants
  • Start-1 week: Remind participants
  • Start-1 day: Make sure recording equipment works and batteries are fresh
  • Facilitate Interview
  • Send sound file for transcription or transcribe it yourself
  • Send out Thank You notes as soon as possible
  • Start+1 week: If transcribing yourself, complete transcriptions
  • Start+2 weeks: Send report to stakeholders for comment
  • Start+3 weeks: Send out final report
  1. Interview Protocol
  2. Post-interview survey (if using one)
  3. Voice recorder with fresh batteries
  4. Telephone adapter if you're doing telephone interviews
  5. Microphone
  6. Extension cord

Pre Interview:

Setting up the Interviews (Facilitator or Administrative Support Staff):

  1. Determine (four weeks before focus group):
    • The program’s details: be as familiar with the program as possible
    • The program context: this ensures that follow-up questions elicit accurate, rich information
    • The purpose
      • Meet with the client(s) to determine their wants and needs
        • Quality measures?
        • Expectations?
        • End result?
        • Needs?
        • How does the client wish to be communicated with?
      • Refer to grant or evaluation plan to align with client desires
    • The participants: who should participate, how many, etc.
    • The location
    • The date(s) and time(s)
    • The incentive, if any, to be offered for participation
    • The number of facilitators needed. If more than one, account for reliability and validity issues
    • Necessary resources: voice recorder, microphone, extension cord, project information sheets, etc.
  2. Set up the:
    • Location
    • Time(s)
    • Incentive avenue. Follow-up with accounting to make sure that stipends or gift cards are purchased and the proper procedures are being followed.
  3. Invite participants (three weeks before interview(s)):
  4. Gather the necessary items the day before the interview(s):
    • Interview Protocol
    • Post-interview survey (if using one)
    • Voice recorder with fresh batteries
    • Telephone adapter if you're doing telephone interviews
    • Microphone
    • Extension cord


Development of the Protocol (Evaluator and/or Facilitator): [1]

  1. Review the client needs, grant deliverables, and evaluation plan to determine the research question the interview should answer. Corbin & Strauss (2008)[1] recommend not only determining the research question but:
    • Developing sensitivity to the data. They recommend that your task is not to be objective but to draw upon your background and past experience in order to understand what the data are saying (p. 33).
    • Using the literature. They recommend reading technical and non-technical literature but not to the point that the literature stifles the qualitative researcher (p. 36).
    • Deriving theoretical frameworks. A theoretical framework is a conceptual outline of the current literature on a topic. Corbin and Strauss (2008) prefer not to begin a research study with a predefined theoretical framework, but they concede that a theoretical framework helps the researcher build on or refute current literature. Be aware of the literature on your topic well enough to know where your study diverges from the theoretical framework.
  2. Draft the following as the introduction to the interviews:
    • Facilitator introduction
    • Welcome and thanks
    • Purpose of the focus group
      • Tell them why they were picked
      • Explain the research question
      • Address the grant or evaluation plan
    • Confidentiality and permission to record
  3. Start protocol with the evaluation questions and develop probes for each question
    • Question and probe characteristics:
      • Are they in logical order?
      • Are they open-ended?
      • Will they draw out conversation?
      • Are they leading?
      • Will they provide possible extra information?
      • Do they elicit honesty?
      • Do they address both accelerators (positives) and barriers (negatives)?
      • Do they allow for extraneous comments?
      • Are the questions/materials appropriate for the audience and the length of time allowed?
  4. Re-check protocol to make sure it aligns and answers the needs in the grant and the evaluation plan
  5. Review the protocol with the client to insure that it meets their expectations
  6. Practice the flow of questioning and familiarize yourself with the questions and the probes

Interview:

During facilitation of an interview, the facilitator should:

  1. Have strong knowledge of the project – know the purpose of the study and have an outline of the questions that need to be asked/covered during the session.
  2. Keep the conversation on track. Kiernan, Kiernan, & Goldberg (2003) [2] offer a set of phrases to help facilitate the interview.
  3. Know when the participant is getting off topic and refocus if necessary.
  4. Refocus repetition in responses by:
    • Probing deeper into the conversation to learn more about underlying issues
    • Changing the course of the conversation and move the conversation in a different direction or move on to the next topic/question.
    • Listening intently for the participant's word choice. For example, if a teacher participating in an interview says, "Unless at least six students sign up for a course, the teacher will not get paid to teach the course." You may ask why the teacher would say, "the teacher will not get paid to teach" as opposed to something like, "the course will not be offered."
  5. Begin thinking about what the final report will look like?
    • How will the information you’ve gathered be organized or “chunked” into sub-categories?
    • Think about the categories that emerged from the conversation and create labels for those categories.
    • Capture this in a journal kept during the interviews to understand how your thinking is being shaped and re-shaped by each interview.

Post Interview:

After the session (Facilitator and/or Evaluator):

  1. General Housekeeping
    • Write thank-you notes or emails to the participants for their time and valuable comments
    • Download audio files
    • Transcribe files if resources are available
  2. Analysis (using a tool such as Atlas.ti, Nvivo, or MAXQDA)
    • Import transcribed text
    • Code "chunks" of text
      • Using pre-defined categories
      • By creating coding categories as you read each transcript. Corbin & Strauss (2008) recommend beginning the process with a microanalysis which they define as a "detailed coding around a concept. A form of open coding used to break data apart and to look for varied meanings of a word or phrase" (p. 46)[1] It is not feasible to sustain such a detailed microanalysis throughout the entire set of transcripts. A coding scheme will emerge from the analysis.
      • As you define each code, check it against multiple quotes and against your other codes. Your codes will changes as you go deeper into the analysis. Some will fall naturally into coding hierarchies.
      • At about one-fourth of the way through the transcripts, conduct an analysis as if it is the end of your report (this recommendation comes from Ray Maietta, Ph.D -- www.researchtalk.com). Here are some questions to consider:
        • How well do your quotes fall into your categeories?
        • What quotes do not fit? Do these have anything in common?
        • Consider using the paragraph as your unit of analysis and categorize paragraphs or fairly large chunks of information instead of smaller chunks such as an utterance within a sentence. This will help later when you analyze your codes for relationships.
        • Organize your quotes graphically to see how they relate to each category, how they overlap from one category to another, and to determine how well each quote fits within the category. If appropriate, organize them into families of categories.
        • Notice where two or more categories converge around a set of quotes. What does this tell you about your data? Did you expect to see these categories converge?
        • If your software allows, create a report showing how your coding categories are distributed across each transcript. Have you coded one transcript more heavily in some categories than others? Is that warranted?
        • If you are stuck with certain quotes, use the memo feature to capture your reflections on how you might categorize the quote or leave it out altogether. This may later help you identify limitations of your analysis or compare your thinking with other coders.
      • Continue coding the remaining transcripts and continue repeating these steps.

Report Writing (Facilitator and/or Evaluator):

  1. Decide on the format of your report
    • General format:
      • Executive Summary (comes first, but written last)
      • Method
      • Comments and text based on questions and response groupings
      • Limitations
      • Themes
      • Recommendations
  2. Write report sections
    • Review themes already outlined
    • Outline and pick comments that express the sentiment of the themes and the mood the best
  3. Outline executive summary
    • What do we actually want to write in the summary?
    • What themes should be represented?
    • Any quantitative numbers?
    • Any qualitative data?
  4. Write executive summary

References

  1. ↑ 1.0 1.1 1.2 Corbin, J. & Strauss, A. (2008). Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Third Edition. Thousand Oaks: Sage Publications.
  2. ↑ Kiernan, M., Kiernan, N.E., & Goldberg, J. (2003). Using Standard Phrases in Qualitative interviews. Tipsheet #69, University Park, PA: Penn State Cooperative Extension. Available at http://www.extension.psu.edu/evaluation/pdf/TS69.pdf.
Retrieved from "http://www.evaluationwiki.org/index.php/General_Interview"
Personal tools