| In 1993, the Manpower
Demonstration Research Corporation (MDRC) began a 10-year evaluation of
the Career Academy approach — a widely established school reform
initiative that encompasses the key principles of the national school-to-work
(or school-to-career) movement. As such, the primary goals of the Career
Academy approach are to improve students’ performance in high school
and to provide them with clearer pathways to post-secondary education
and careers. While attempting to create more supportive teaching and learning
communities within high schools, Career Academies also seek to integrate
academic and vocational instruction and to provide work-based learning
opportunities for students, with the aim of preparing them for their lives
beyond high school — whether they are going straight into the job
market or planning to attend college first.
This report is the
second in a series from MDRC’s Career Academies Evaluation, which
focuses on 10 high schools and their Career Academies from across the
country. The first report — Career Academies: Early Implementation
Lessons from a 10-Site Evaluation — described the 10 Career Academies
participating in the study and their local contexts. The current report
begins to look inside the participating Career Academies and focuses on
the extent to which they function as "communities of support"
for students and teachers. For students, such support includes the personalized
attention they get from their teachers, their teachers’ expectations
of them, their fellow classmates’ level of engagement in school,
and the opportunities they have to collaborate with their peers on school
projects. Teachers are supported by, among other things, opportunities
for professional collaboration and development, adequate resources, the
capacity to influence instructional and administrative decisions, and
opportunities to give personalized attention to students. Both this study
and previous research have identified these dimensions of support as factors
that can have an important effect on both students’ motivation and
engagement in school and teachers’ job satisfaction and sense of
whether they are making a difference in their students’ lives.
The key findings reported
here indicate that the Career Academies provide their students and teachers
with a greater degree of institutional and interpersonal support than
is available to their non-Academy counterparts in the same comprehensive
high schools. Students in the early stages of their Academy experience
report that they are somewhat more motivated to attend school and that
their schoolwork seems more relevant to their future education and career
goals. At the same time, while Academy students appear to be highly engaged
in school, they do not appear to be more engaged than their non-Academy
counterparts. Academy teachers were more likely to see themselves as belonging
to a strong professional community and indicated higher levels of job
satisfaction than their non-Academy counterparts in the same high schools.
Nevertheless, Academy and non-Academy teachers were about equally likely
to rate themselves as being highly effective with their students.
These early benchmarks
of contrast between the Academy and non-Academy school environments, as
viewed through the eyes of their students and teachers, should be interpreted
with caution. Most important, the current report focuses on only a limited
set of student experiences that are likely to be affected by the Career
Academies. Many of the students participating in the study had gone through
only one high school year at the time the data for this report were collected,
and only about one-third of the students had reached their second year
in the study. Thus, they had little or no exposure to some of the key
Academy components, particularly its integrated academic/occupational
curricula and work-based learning opportunities. Future reports will examine
a broader set of indicators of student performance and engagement in school
and at work, and will capture the cumulative effects, if any, as they
accrue through high school and beyond.
What
Is a Career Academy?
Career Academies are
organized as "schools-within-schools" in which groups of students
(usually 30 to 60 per grade in grades 9 through 12 or 10 through 12) take
several classes together each year with the same group of teachers. The
goal of the school-within-a-school is to promote more constructive relationships
between and among teachers and students and thereby to increase students’
engagement and success in high school. Each Academy focuses on a career
theme, such as business and finance, electronics, or health occupations,
to provide opportunities for teachers and students to integrate academic
and occupation-related classes in an effort to enhance their relevance
to the real world while preserving academic rigor. Academies also forge
partnerships with local employers from a particular field to help plan
and guide the program, and to serve as a source of adult mentors and work
internships for the students.
While Career Academies
have existed for over 25 years, the approach has gained greater prominence
recently as states and school districts have increased their efforts to
restructure high schools. This restructuring is aimed at supporting students
academically while providing them with marketable skills, work-based learning
experiences, and smoother transitions to post-secondary education and
productive employment. Interest in Career Academies was further accelerated
with the passage of the federal School-to-Work Opportunities Act in 1994.
The Act provided federal funding and support for states and localities
to take a systemic approach to helping schools forge stronger partnerships
with their communities and with local employers, and to create opportunities
for students to begin making connections between schooling and their career
aspirations. The core components of the Career Academy approach reflect
many of the cornerstones of the new legislation and its objectives as
well as many key dimensions of other reform efforts to improve high schools.
While there are no reliable data on the total number of Career Academy
programs nationally, current estimates suggest that Career Academies have
been established in at least 600 to 700 high schools.
The
Career Academies Evaluation
Ten high schools
and their Career Academies, representing most of the major, established
networks of Career Academies across the country, are participating in
this study. Their names, locations, and affiliations are shown in Exhibit
1. Most of the nine school districts in the evaluation (one district
includes two of the participating Career Academies) are in urban areas
or small cities and enroll substantial percentages of black and Hispanic
students compared with national averages. The participating school districts
also have, on average, higher dropout rates, unemployment rates, and
percentages of low-income families. Most Career Academies across
the country are located in such districts, and MDRC purposely sought such
sites for the Career Academies Evaluation. Each of the participating Career
Academies attempted to serve a wide range of students, including those
who were doing well in school and those who appeared to be at risk of
dropping out.
As they entered
the study, each of the 10 sites had already established the basic Career
Academy components mentioned above: a school-within-a-school organization,
an integrated academic/occupational curriculum, and partnerships with
local employers. This combination of features was not available elsewhere
in the participating high schools. Exhibit 1 indicates that the Career
Academies in the study reflect a range of occupational themes: three are
in the business and finance fields; three focus on high-technology areas
such as electronics and aerospace technology; and one each is in the fields
of health occupations, public service, travel and tourism, and video technology.
The Career Academies
Evaluation will follow nearly 2,000 students from the 10 sites through
their high school years and for up to three years following their scheduled
graduation from high school. Each of these students was identified by
the participating Career Academies as eligible for participation in their
programs. Because each of the programs received applications from more
students than they could serve, however, approximately 1,100 of these
students were randomly selected to enroll in the programs; the remaining
students were not selected and chose, or were assigned, to attend other
regular high school programs. This random assignment research design
provides a unique foundation for an unusually rigorous assessment of the
Academies’ effects on student outcomes.
This report draws
on students’ and teachers’ responses to questionnaires they
completed during their first or second year in the study. From the full
sample, 1,406 students and 468 teachers in the Academy and non-Academy
environments of the 10 high schools completed a questionnaire. They were
asked a range of questions about their experiences, behaviors, and attitudes
in school, and their responses were then used to create indicators of
factors that were likely to affect the students’ engagement and performance.
Most of the analyses conducted for this report focus on the proportion
of students or teachers who gave consistently "high" ratings
across the groups of questionnaire items. In general, a "high"
rating indicates that a given respondent consistently and strongly affirmed
a group of statements about his or her experiences, behaviors, and attitudes.
The report also draws on qualitative information collected during on-site
interviews with teachers and students and observations of
Academy classes and activities.
It is important
to note that there were no systematic differences in the background characteristics
of the Academy and non-Academy students who completed the questionnaire.
This increases the confidence one may have that any differences that emerge
based on the questionnaire data can be attributed to the difference between
the Academy and non-Academy environments. Nonetheless, the results
should be interpreted somewhat cautiously because there may be some unmeasured
differences between the background characteristics of Academy and non-Academy
students who completed a questionnaire. Also, there were some modest differences
between the characteristics of the students who completed a questionnaire
and those students who did not.
Academy and non-Academy
teachers who completed the questionnaire had several similar key background
characteristics. However, some measured differences emerged for other
characteristics, for which statistical controls were applied in the analysis.
Thus, while differences between Academy and non-Academy teachers that
emerge from the questionnaire data may be partly attributable to
the Academies, they may also be the result of some underlying differences
in the unmeasured characteristics of the two groups of teachers. In general,
the findings from the comparison of Academy and non-Academy teachers is
intended to highlight differences in the teaching and learning opportunities
being made available to Academy and non-Academy students.
The Career Academies
Evaluation is being supported by the U.S. Departments of Education and
Labor, 15 private funders, and the 10 sites participating in the study.
Career
Academies as Communities of Support
Where the first
report from the evaluation was concerned with the structural elements
of the Career Academy approach as they had been implemented and sustained
in the 10 sites, the current report takes an early look inside the Academies
to shed light on the experiences of their students and teachers and to
contrast these experiences with those of their peers in the participating
high schools. In particular, the report focuses on whether the Career
Academies serve as "communities of support" for students and
teachers. In the context of this report, such communities provide a range
of institutional and interpersonal supports that aim to enhance student
motivation and engagement in school and to help teachers optimize their
students’ learning experiences.
Exhibit
2 shows two simplified conceptual models of how selected institutional
and interpersonal supports are hypothesized to affect student engagement
and teacher effectiveness, respectively. For students, the Career Academy
approach (particularly its school-within-a-school organization) is hypothesized
to enhance support from teachers, peers, and parents which will help motivate
them to do well and lead to greater engagement. Similarly, the Career
Academy approach is hypothesized to offer teachers institutional supports
(such as teacher collaboration, resources, and ability to influence work),
which enhance certain interpersonal supports and, ultimately, increase
their job satisfaction and sense of effectiveness. Measures of the constructs
in Exhibit 2 were created from the groups of student
and teacher questionnaire items described above.
All of the differences
between Academy and non-Academy participants discussed below are
statistically significant, except when noted otherwise. Statistically
significant differences are those that are least likely to be due to
chance.
How
Do Career Academies Support Students?
- Career Academies increased,
at least modestly, the support students receive from their teachers
and peers.
The first set of
bars in Exhibit 3 indicates that Academy students
were more likely than their non-Academy counterparts to report that their
teachers give them personalized attention and have high expectations of
them. The second set of bars shows that Academy students were also
more likely to see their classmates as being engaged in school
and to work with them on school projects and assignments. In general,
these findings suggest that the structural features of the Career Academy
approach — particularly, in this case, the school-within-a-school
organization — offer students a greater degree of support from teachers
and peers than is available to similar students in the regular high school
environments.
- These supports appear to
have enhanced student motivation and increased the connections they
see between what they are learning in school and their longer-term education
and career interests.
The third set of
bars in Exhibit 3 indicates that Academy students
were slightly more likely than non-Academy students to indicate that they
were intrinsically motivated to attend school. That is, these students
reported that they attend school primarily because they like it and are
interested in what they are learning, rather than because they must or
primarily in response to external pressures or the negative consequences
of not attending. The fourth set of bars indicates a further difference
between Academy and non-Academy students: Academy students were
more likely to perceive a strong connection between what they are learning
in school and their longer-term education and career interests.
- Although most Academy students
indicated that they were highly engaged in school, they were no more
likely to do so than their non-Academy peers.
In this analysis,
"engagement in school" includes students’ self-reported
behavior (such as being prepared for and paying attention in class, exerting
effort in class, and doing homework), emotional state when in school,
and psychological commitment to doing well in school. Further analyses
of the questionnaire showed that, for both Academy and non-Academy students,
both the interpersonal supports and the motivational processes were related
to this self-reported measure of school engagement. Thus, the Academy
students’ higher levels of support, motivation, and belief
that their schoolwork has future relevance should translate into higher
levels of engagement in school. However, as indicated in Exhibit
3, the questionnaire data indicate that, at this early point in the
follow-up period, the Academy and non-Academy students reported similar
levels of behavioral, psychological, and emotional engagement. In all,
about two-thirds of both Academy and non-Academy students were highly
engaged in school, indicating that students were likely to be engaged
in school at this point in their high school careers even if they were
not in the program.
At the time the
data for this report were collected, most students in the study sample
had little exposure to the integrated academic and occupation-related
curricula, had not yet participated in work-based internships, and had
only a few opportunities to participate in activities that would prepare
them for post-secondary education and careers. Thus, it is too early to
tell whether the full Career Academy program will have a cumulative effect
on student engagement and performance.
How
Do Career Academies Support Teachers?
- Academy teachers indicate
that they have more opportunities to collaborate with colleagues and
to influence decisions in key areas of their work.
The differences
shown in the first three sets of bars in Exhibit 4
indicate the extent to which Career Academy teachers are exposed to a
variety of institutional supports that are not as widely available in
the regular high school environment. Resources include materials such
as books and supplies, as well as nonmaterial resources including time
and spaces to get together with colleagues. The teacher collaboration
construct captures the degree to which teachers meet regularly to discuss
instructional strategies, student-related issues, and curriculum integration.
Areas of work over which teachers report a high degree of influence include
instruction-related areas such as determining the content of professional
development activities and selecting books, materials, and course content,
and administrative areas such as disciplinary policies, elements
of the daily schedule, and selecting students for their classes. Career
Academy teachers were more likely than their non-Academy peers to give
high ratings to each of these dimensions of institutional support. These
variables are important because they are likely to be most sensitive to
organizational and policy changes reflected in the Career Academy approach.
- Career Academy teachers were
more likely than non-Academy teachers to indicate that they were part
of a strong teacher learning community and that they emphasized personalized
attention to their students.
The fourth set
of bars in Exhibit 4 shows that Academy teachers
were more likely than their non-Academy counterparts to give a high rating
on the indicators of a strong teacher learning community. Key dimensions
of a strong teacher learning community include whether teachers have opportunities
to enhance subject matter knowledge and instructional strategies and to
continue their professional growth. They also include indicators of whether
teachers work closely with colleagues who are continually learning and
seeking new ideas and who support their efforts to develop professionally.
The fifth set of
bars in Exhibit 4 indicates that Career Academy
teachers were more likely than their non-Academy colleagues to place a
high level of emphasis on personalized attention to students. This indicator
includes the extent to which they try to be accessible to students, go
out of their way to help them both academically and personally in school,
and take an interest in students’ achievements and concerns beyond
the classroom.
- Career Academy teachers expressed
higher levels of job satisfaction than non-Academy teachers.
Academy teachers
were more likely than non-Academy teachers to indicate a high level of
satisfaction with their jobs (see Exhibit 4).
For the purposes of this analysis, teachers’ satisfaction with their
jobs includes the extent to which they were satisfied with the school
learning environment, the intellectual challenge, and the enforcement
of disciplinary policies. It also includes teachers’ satisfaction
with their work overall and whether they are likely to continue
teaching.
- Academy and non-Academy teachers
were about equally likely to report a high level of effectiveness.
As shown in Exhibit
4, about half of the Career Academy teachers indicated that they felt
they were highly effective in their jobs, compared with 44 percent of
the non-Academy teachers — not a statistically significant difference.
This indicator is intended to capture teachers’ sense of the extent
to which they believe that they can control or influence their students’
success in school and their sense of whether they are, in fact, making
a difference in their students’ lives. Further analysis did not reveal
any systematic differences between Academy and non-Academy teachers in
the individual components of the self-reported effectiveness
measure.
In short, Academy
teachers’ enhanced support, emphasis on personalized attention to
students, and job satisfaction do not appear to have translated into systematic
increases in their sense of effectiveness. One hypothesis for explaining
this pattern of findings is that Career Academy and non-Academy teachers
may have differing definitions of "effectiveness." In
other words, Academy students may be performing better than non-Academy
students, but the Academy teachers may be expecting more from their Academy
students and from themselves. The self-reported measure of teacher effectiveness
used in this analysis may mask differences in other measures of effectiveness
such as students’ assessments of their school experiences or assessments
of student success indicated by school records. Also, the lack of difference
in Academy and non-Academy teachers’ ratings of personal effectiveness
should be interpreted cautiously because the measure does not take into
account the possibility that Academy teachers may consider a greater number
of performance dimensions as relevant. For example, interviews with Career
Academy teachers revealed that many of them felt they should provide Academy
students with a range of interpersonal and employability skills as well
as academic skills.
Finally, although
the enhanced supports and satisfaction that Academy teachers experience
may be necessary conditions for greater effectiveness, they may not be
sufficient. Academy teachers highlighted several significant challenges
that must be met to attain the ideals represented by the Career Academy
model. For example, many teachers focused on the difficulty of
integrating a rigorous vocational curriculum with a relevant and demanding
academic curriculum and ensuring that the Academy’s work-based learning
component provides students with exposure to the real world and a rich
learning experience.
Exploring
Further Implications of the Findings
Findings from the
first two reports from the Career Academies Evaluation suggest that the
approach holds some promise for restructuring high schools with the aim
of creating more supportive teaching and learning communities for students
and teachers. Each of the sites in the study has demonstrated the feasibility
of implementing and sustaining the core structural elements of the Career
Academy approach and adapting them to their local needs and circumstances.
The first report highlights some of the challenges involved in undertaking
this restructuring. The current report indicates that, within the confines
of these new structural arrangements, Academy students and teachers experience
a greater degree of institutional and interpersonal support than do their
peers in the regular high school environments. This enhanced support appears
to have translated into somewhat higher levels of motivation among Academy
students and higher levels of job satisfaction among Academy teachers.
However, while students in their initial years in the Academy programs
appear to be highly engaged in school, they are no more engaged than their
non-Academy peers.
While supports
discussed in this report may provide some of the necessary conditions
for improving students’ performance in high school and guiding them
toward clearer pathways to post-secondary education and careers, it is
still too early to tell whether the overall Career Academy experience
is sufficient to accomplish these goals. At this early stage in the evaluation,
very few of the students had participated in the Academies’
work-based learning activities (which usually occur during or just after
the 11th-grade year), and students’ exposure to the occupational
theme and integrated curricula was still relatively limited. Also, as
the types of supports noted earlier continue over two or three years,
they could have a cumulative effect on student engagement and other outcomes
such as progress toward graduation and advancement to higher-level academic
and occupation-related courses. As a result, it is critical to follow
students further through their high school years and beyond. Future reports
from the evaluation will examine students’ exposure to the Career
Academies’ integrated curricula and work-based learning activities
and will include longer follow-up periods to examine the potential cumulative
effects the Academies may have on students. Specifically, future reports
will provide findings on whether the Career Academies are producing impacts
on additional measures of student engagement in high school, their progress
toward graduation, and their transition to post-secondary education and
work.
It is also important
to note that these findings have been aggregated across all the sites
and all students. These aggregated findings may mask differences among
the sites that are associated with particular strategies for utilizing
the basic elements of the Career Academy approach to support students
and enhance their engagement in school. They may also hide differences
among the participating high schools and school districts that could enhance
or limit the supports available to Academy or non-Academy students. Future
reports from this evaluation will explore variation across
sites on a number of measures of program and contextual differences and
their potential effects on students.
Finally, these
aggregated findings may obscure the fact that some subgroups of students
may benefit more (or less) from the Career Academy experience than others.
Future reports will also examine the effects Career Academies have on
subgroups of students who are defined by background characteristics that,
for example, are associated with a risk of poor performance in high school
or that indicate a high level of prior school engagement.
|
