Data collection doesn't always have to be about creating a survey. In many cases, it can be as simple as finding a structured way to evaluate the things that are happening around you every day. Below are a few ways to break out of the survey bubble:

Portfolio: A collection of artifacts or work that provide evidence of student learning or program improvement.

Strengths:

  • Shows progress over time
  • Reflective in nature (encourages reflective learning)
  • Provides deep examples
  • Multidimensional (shows learning in different ways)
  • Provides both individual and program-level feedback
  • Provides both numbers and descriptive information

Challenges:

  • Requires planning ahead (pre-determined outcomes, criteria for meeting outcome, experiences to be included, type of reflection, rating tool)
  • Takes time to implement and see progress
  • Need trained evaluators
  • Need system of collecting portfolios (electronic, hard copy)
  • Depending on technology resources, combining aggregate data can take time

Resources needed:

  • Do you have outcomes, criteria, learning experience, and reflection prompts prepared?
  • Do you need to train evaluators?
  • Do you have a system for collecting portfolio materials?
  • Do you have time to look through portfolios and analyze evidence?


Observation: A systematic method of collecting data through unobtrusive visual means (e.g., watching people or places) in order to collect information.

Strengths:

  • Unobtrusive – does not require participant engagement
  • Requires seeing beyond nature perspective
  • Often effective with physical plant and watching for student trends
  • Useful for gathering initial data to couple with survey or focus group
  • Provides both numbers and descriptive information

Challenges:

  • Requires planning ahead (e.g., protocols, charts, journals)
  • Non-responsive in nature
  • Limited in the type of data it can collect
  • Need trained observers
  • Need system of collecting information

Resources needed:

  • Do you have a protocol?
  • Do you need to train observers?
  • What is your timeline?


Document Analysis: A form of qualitative research in which documents are used to give voice, interpretation and meaning. Any document can be used, common documents may be: application materials, student newspaper or publications, marketing materials, meeting minutes, strategic planning documents, etc.

Strengths:

  • Documents are readily available
  • Documents are already collected or easily collected
  • Low costs
  • Documents are a stable data source (they don’t change)
  • Can be collected on a quick timeline

Challenges:

  • Non-responsive in nature
  • Documents are context and language specific
  • Documents are often disconnected from their creator
  • All documents are written through a lens, need to be aware of lens in order to assess objectivity
  • Data analysis takes time

Resources needed:

  • How do you gain access to the documents?
  • Do you know how to set up a coding system?


One-Minute Assessment: Very short assessments of what a participant is “taking away” from their experience. Should be targeted at a specific learning or program outcome.

Strengths:

  • Provides a quick summary of take away from student perspective
  • Quickly identifies areas of weakness and strengths for formative assessment
  • Can track changes over time (short-term)
  • Non-verbal (provides classroom feedback from all students)
  • Captures student voice
  • Short time commitment 
  • Provides immediate feedback

Challenges:

  • Non-responsive
  • Short (so you may lose specifics)
  • Sometimes hard to interpret
  • Need very specific prompts in order to get “good” data
  • Plan logistics ahead of time and leave time during program/course
  • May need to be collected over time

Resources needed:

  • Do you have a strong prompt?
  • Have you reserved time to collect data?
  • Do you have a system for collecting data in a non-rushed manner?


Visual Methods: Captures images as a main form of data collection, usually also includes captions or a journal to accompany images. Most often used for photo journals, video projects, and visual art projects.

Strengths:

  • More detail and depth to data
  • Visual aspect allows for depth in sharing results
  • High levels of student investment
  • Can use images captured for multiple uses
  • Very descriptive in nature

Challenges:

  • Beware of threats to alterations of images (especially with technology)
  • Usually smaller number of perspectives
  • Time for implementation and follow-through
  • Analysis takes time
  • Resources may be needed in order to capture images

Resources needed:

  • How will your participants capture images (resources)?
  • What prompt will you use to make sure participants have a clear direction?
  • Do you have time to gather and process information in your timeline?
  • Have you accounted for time for member checking?

Case Study: A form of qualitative descriptive research, the case study looks intensely at an individual, culture, organization or event/incident.

Strengths:

  • More detail and depth to data
  • Multiple perspectives are gathered
  • Tells a story
  • Very descriptive in nature

Challenges:

  • Takes significant time to gather information and analyze
  • More perspectives = more time
  • Narrow purpose as far as sharing data afterward
  • Analysis takes time
  • Resources may be needed in order to capture data
  • Not meant to be generalizable but can be transferrable

Resources needed:

  • How will you capture data?
  • Do you have a clear understanding what you are profiling and why?
  • Do you have time to gather and process information?
  • Have you allocated time for member checking?
Have more questions? Submit a request