School Performance Framework Deep(er) Dive: Part 1 – Overview

Data nerds in Oakland rejoiced last November when the Oakland Unified School District released School Performance Framework reports for 2015-16.  After a soft launch internal to the District in 2014-15, this was the first public release of the School Performance Framework (SPF) – a snapshot of how schools are doing. We now have an easier, visual, way to understand the current state and trajectory of OUSD schools, with a plan to add all other Oakland public schools (i.e. charters) this coming year.

Why does the SPF matter?

The SPF is a HUGE step to making Oakland school data more visible and understandable to the general public. Like the stars system on Yelp or report cards for students, it puts relevant information into the hands of people who need to know in an easily digestible and visual form.

SPF Tiers

What’s special about the SPF?

When thinking about school quality, many people tend to gravitate to a single measure: results on standardized tests. Alternatively, word of mouth, and therefore often socioeconomically defined networks of friends and acquaintances, can define reputation more than any objective set of measurements. Both of these extremes are too one-dimensional, or worse, misleading.

The new School Performance Framework from OUSD is a much more equitable, comprehensive, and developmental.

For example:

  • It looks at both whole school populations AND subgroups that are often left behind in classrooms, such as low-income or African-American students. This highlights achievement gaps between groups of students that might not be immediately evident.
  • It captures a number of different factors that contribute to school quality. The SPF includes metrics related to both academic (state testing, reading levels, etc.) and climate (suspensions, EL reclassification rates, etc.).  Hooray for a multi-measure approach! All the metrics are scored and weighted, resulting in an aggregate score from 1 to 6 (rounding down to get the tier).
  • It considers both absolute performance AND growth. Knowing how much a school has improved is critical for highlighting historically lower-performing schools that are now making huge strides for students.

The overall result? The following bar chart shows the tier by tier comparisons for the soft launch year and last year. We see some mild progress overall, with a 33% decrease in the lowest tier, and a 33% increase in schools in the middle or yellow Tier 3. Tiers 2, 4, and 5, all essentially remained steady. My upcoming posts will delve into the details behind the movements up and down the scale for individual schools, and then what is underlying that in terms of student subgroups and correlation with OUSD programs.

OUSD SPF Scores from 2014-15 to 2015-16

 

By the way, Oakland has been leading the way on this. In September, the CA State Board of Education adopted an evaluation rubric for schools to replace the old Academic Performance Index (API) that is also multi-measure, includes growth, and absolute performance and looks at subgroups…sound familiar?!

Next up: Who’s improving…and who’s getting left behind? (Looking at 2015-2016 SPF results)

### My name is Carrie Chan, and I’m Educate78’s newest staff member. I joined the organization as an Analyst, and I LOVE data (feel free to call me a data geek). As a former OUSD student, I also care a lot about Oakland public schools. That is why I am so excited about this new blog series, “Crunched!” which will take a data-driven approach to important, relevant questions facing Oakland public schools. Please email me with ideas or requests.

SBAC Oakland & Equity, Part 3

CRUNCHED!

A data-based look at Oakland public schools

Happy New Year! We are back with the last post in our three-part blog series about 2016 SBAC results for subpopulations in Oakland public schools. This post is focused on Latino and African American students, two groups of students who historically have been underserved.

Hispanic/Latino students make up biggest ethnic subgroup in Oakland – 45% of students in public schools citywide. The following schools serve more than 80% Latino students and outperform the district average.

Hidden Gems: Schools with High Latino Population (>80% of Student Body) whose Latino Population Outperformed Citywide Average (sorted based on highest aggregate differential between school’s ELA and Math scores and citywide average)

Have to offer kudos once again to Acorn Woodland, whose Latino students outperformed the citywide average for all students in both ELA and Math! This was the only OUSD school to achieve this “two-fer”, though Life did in ELA.

The Latino students at Lighthouse High and Oakland Unity High far outperformed not only the city average in both ELA and Math, but out-performed the statewide averages for all students (59% in ELA, 33% in Math)! Families who consider leaving Oakland because they believe the high schools are bad are missing out on these two high schools.

African American students comprise about 25% of Oakland’s public school student population, and have also historically been underserved. Here are five schools in which African American students outperform the city average:

Hidden Gems: Schools with High AA Population whose AA Population Outperformed Citywide Average (sorted based on highest aggregate differential between school’s ELA and Math scores and citywide average)

The performance of COVA’s African American students in ELA is the 30th best among the 129 public schools in Oakland. Maybe it’s something to do with giving students a well-rounded education that includes the music and arts?!

Many of the schools listed in this post today don’t get named in the usual lists of academic high performers but I believe they deserve recognition for the demonstrated quality of education they are providing to groups of students who are historically underserved, not just in Oakland but across the country.

Absolute performance is just one measure of academic progress, but growth in SBAC scores will be an equally insightful measure on student progress, as well as inspecting the achievement gap in schools to see if schools are serving all students equitably. To be continued in future posts!

###

My name is Carrie Chan, and I’m Educate78’s newest staff member. I joined the organization as an Analyst, and I LOVE data (feel free to call me a data geek). As a former OUSD student, I also care a lot about Oakland public schools. That is why I am so excited about this new blog series, “Crunched!” which will take a data-driven approach to important, relevant questions facing Oakland public schools. Please email me with ideas or requests.

SBAC Oakland & Equity, Part 2

CRUNCHED!

A data-based look at Oakland public schools

Welcome back to our continued examination of SBAC data of Oakland schools. In this post, we’ll look at the schools where English Learners (EL) represent more than 50% of the students and see who is doing well for that subgroup, which across the city comprises 30% of our public school students. Be sure to check out the first blog in this series on why we should look beyond just absolute performance of whole student population.

Hidden Gems: Schools with EL Population of more than 50% of students whose EL Population Outperformed Oakland Average (sorted based on highest aggregate differential between school’s ELA and Math scores and citywide average)

We were disheartened to see that there is only one school in this category. However, huge props to Lincoln Elementary for outperforming the Oakland average in math while serving an oft-underserved population!

In the interest of finding other schools to learn from, we also looked at schools with a high concentration of EL students whose EL population is outperforming the Oakland EL average:

Hidden Gems: Schools with EL Population of more than 50% of students whose EL Population Outperformed Oakland’s EL Average by at least 5 Total Percentage Points* (sorted based on highest aggregate differential between school’s ELA and Math scores and citywide average)

Given how language-intensive the SBAC is (even the Math portion), looking at only this test is insufficient for understanding the progress of our English Learners. It also overlooks important differences within our English Learner population, which includes dozens of different primary languages and range in experience from refugees who have just come from a war-torn homeland, to students born in Oakland who speak a heritage language at home.

If you have ideas on how to better track the progress of our EL students in a way that is reliable and consistent across schools, I’d love to hear from you! Please shoot us an email!

A growing number of schools in Oakland have created newcomer programs to address the influx of immigrants, unaccompanied minors, refugees and asylees. Some of these children have experienced extreme violence and trauma and have had little or no formal schooling. Others speak native languages unfamiliar to any staff member at the school. These schools, like Oakland International High School, may not make it onto a “high performing on SBAC list” but deserve applause for their amazing and dedicated efforts to serve newcomers. We’ll feature more in a future blog post about EL students and newcomer programs across the city (including a dual-immersion middle school that is currently in the works)!  Stay tuned!

Next time: a closer look at schools with a high percentage of Latino and African American students, two subpopulations that often are left behind in classrooms.

*We understand from talking with others who analyze SBAC data that different schools may use different methodologies for even classifying students as EL, so we may be missing some schools that reclassify students more quickly.

###

My name is Carrie Chan, and I’m Educate78’s newest staff member. I joined the organization as an Analyst, and I LOVE data (feel free to call me a data geek). As a former OUSD student, I also care a lot about Oakland public schools. That is why I am so excited about this new blog series, “Crunched!” which will take a data-driven approach to important, relevant questions facing Oakland public schools. Please email me with ideas or requests.

SBAC Oakland & Equity, Part 1

CRUNCHED!

A data-based look at Oakland public schools

Educate78 believes in holding all public schools accountable for equity and quality. One way we do that is through a careful and intentional analysis of data. One important data set that we all need to be looking closely at – educators trying to close the achievement gap, administrators trying to scale up best practices, and parents trying to make the best choices for our kids – is the Smarter Balanced Assessment Consortium (SBAC).

That’s why we’re launching CRUNCHED!, a new blog series that takes a data-driven approach to important, relevant questions facing Oakland public schools, thanks to the hard work of our data analyst Carrie Chan (who is an OUSD alum!). We are excited to share our analysis of SBAC and the implications for Oakland schools. “Happy Data Diving!”

Our first post: SBAC Oakland & Equity, is one of a three-part series examining recent student achievement results. We hope you’ll stay tuned over the next few weeks and beyond to crunch the data with us.

Spring 2016 was the second time that students across California took the Smarter Balanced Assessment Consortium (SBAC) test, which is supposed to be aligned to Common Core standards, which in turn is supposed to represent “internationally competitive” expectations. On October 18th, the California Department of Education finally released all school-level SBAC results from Spring 2016 (an earlier release in August included only 80% of students).  This analysis reflects this latest release of data.

Most often, people focus on student performance for a school as a whole when rattling off statistics.

Based on this definition, let’s look at Oakland’s “top” performers:

Top Public Schools in Oakland

(All schools with 70% of their students meeting or exceeding standards in either Math or ELA or both on Spring 2016 SBAC)

The schools above are listed in order of greatest total differential—who had the highest percentage of students proficient and above in ELA and math, combined, compared to the Oakland school averages (ELA average = 31%, Math = 24%).

But if you’re like us, you want to dig deeper into the make-up of that score. We know that standardized test scores are highly correlated with student demographics, and some of these schools serve populations that are more affluent and are not English Language Learners – and we were thrilled to see that some of these high performers are serving many low-income students of color exceptionally well!

We also know that overall school performance can mask significantly different outcomes for some populations within a school. So when we see this kind of data, we wonder: what percentage of high need students does this school serve? Does this school have an achievement gap? How are low-income students (FRL) doing? What is the performance of English Language Learners (ELL)?

FINDING THE HIDDEN GEMS

We started looking for Oakland schools that are doing a great job serving populations that are normally “left behind.”

To start unpacking the data, first we looked at the performance of just the low-income students by sorting all Oakland schools by their concentration of students in poverty – that is, those that had a very high percentage of students eligible for free or reduced price lunch.  .  (Note: Children in a family of four with an annual income of less $44,863 would qualify for reduced-price lunch and income of less than $31,525 or less would qualify for free lunch.)

In some of these schools, the low-income students outperformed the Oakland city-wide average in either ELA, Math, or both. We call them our “Hidden Gems” because these schools usually don’t get the recognition they deserve for doing great things for our high-need Oakland students.

Hidden Gem Schools: Schools with a High FRL Population (>90%) whose FRL Population Outperformed Oakland Average in ELA or Math.

One special bright spot: ACORN Woodland, which outperforms the entire citywide average in both categories, while serving a historically underserved population.  Not only do they serve one of the highest FRL populations in the city (top 20%), they also serve a student population where 70% of the students are English Language Learners (ELL). We wonder what Acorn Woodland is doing that can be shared with educators in other schools.

MORE WORK TO DO

In a city in which almost three-fourths of public school students are eligible for free or reduced lunch, we all can learn a lot from these schools. And, even these schools have a long way to go to have all their students meeting or exceeding standards – which is, of course, what everyone is striving for. Oakland’s public school enrollment season is about to begin, and we know that parents and guardians need honest, in-depth, and up-to-date school data. We hope this helps parents make informed choices, and schools to promote their successes.”

Up next time: schools with high ELL populations outperforming OUSD’s average.

###

My name is Carrie Chan, and I’m Educate78’s newest staff member. I joined the organization as an Analyst, and I LOVE data (feel free to call me a data geek). As a former OUSD student, I also care a lot about Oakland public schools. That is why I am so excited about this new blog series, “Crunched!” which will take a data-driven approach to important, relevant questions facing Oakland public schools. Please email me with ideas or requests.