Analysis of Data for Meaningful Patterns
by Barry Sweeny, 2003
Once you have collected program evaluation data your next step is to try to determine what it can tell you. That process is essentially one of arranging data to allow comparisons, searching for meaningful patterns in the data, and interpretation or assigning meaning to the patterns found. Of course, those three steps are a simplification, and the actual processand it has a number of steps in it.
INDEX:
1. Assemble all the data that are relevant to the success
of participants in the program. Try to use:
ï At least 3 to 5 years of trend data.
ï Hard data (such as attendance, demographics, test scores, etc.)
ï Soft data (opinion data such as from surveys, observations, focus groups, or interviews).
ï Data which address program goals and objectives, participant growth, implementation
of planned activities, etc.
ï Data which describe both mentor and protege knowledge and behaviors on relevant
topics
2. Display these data so that any trends are apparent and
summarized in a chart. Use one page per topic. Data must be displayed so it reveals
patterns because the data has no inherent meaning by itself. It is only if a pattern
can be found in the data that meaning can be assigned, and then only those like the
staff can appropriately assign the meaning since only they know what may have contributed
to making the pattern. Of course, this is best done by a computer if you can input
the data into a spread sheet.
3. Write a narrative "topic" conclusion stating
what the trends and other data are interpreted to tell you about:
ï Mentor and protege growth, strengths, and needs
ï Program progress, strengths and needs.
Be sure to consider the possible factors that may explain why the data look as they
do, such as changes in staff
training, instructional materials, media use, forms of assessment, etc. which may
have had an impact on learning.
4. Data come in many different forms, such as percentiles, quartiles,
lists of statements, and meets/exceeds/does not
meet. Convert the form each of the data are in to ONE common form. I recommend
using the ìmeets/exceeds/does not
meetî form. Here is how to do this conversion.
Regardless of the original form of the data...
ï If the score or data level is ìsatisfactoryî, write it as ìmeetsî our expectations
(M).
ï If the score or data level is ìnot satisfactoryî, write it as ìdoes not meetî
our expectations. (DNM)
ï If the score or data level is significantly ìbetter than satisfactoryî, write it
as ìexceedsî our expectations (E).
5. Write the translation results (M/E/DNM) into a data chart listing indicators (factors) along one side and symbols for each.
6. Look at each data chart to see if a pattern is evident. If so, write a narrative conclusion describing and explaining the pattern.
7. Give a descriptive title to each of the data summary charts
and the written conclusions. Link the written conclusions and
the data summary charts by placing them on the same page together, or at least creating
a cross-reference so these two items
are clearly related to each other.
8. Make a ìHard & Soft Data Comparison Chart"
to collect the converted data (M/E/DNM) on a single
page. Write a title at the top, topics along one side and data sources along the
other. Enter the M/E/DNM in the right places.
9. Write a narrative conclusion about the patterns found for
each of the ìComparison Chartsî you have done.
10. Identify which indicators are the targets for improvement.
Again, use titles and cross references to ensure that the
appropriate data comparison and written conclusion are linked to each other.
All information on this web page is copyright protected under USA and international laws, © 2003, by the International Mentoring Association, http://www.mentoring-association.org
However, you MAY copy this page, but only under the following conditions:
Home page - General Info - Nonmember Resources - Member Resources - Annual Conference - Membership - Goals - Board