Are Oakland students making progress meeting state standards?

The budget is on everyone’s mind. So are the teacher contract negotiations. (Or maybe just winter break…) At times lost in the mix of all these serious issues is a central question for all Oakland public schools: Are our students making progress meeting state standards in math and language, the building blocks of learning? Well, here’s a table showing Oakland’s progress on the recently released state SBAC tests from earlier this year on math and ELA (English Language Arts):

Chart comparing Oakland proficiency rates on SBAC ELA and Math compared to statewide proficiency rates.

The answer: Technically, yes. Nevertheless, we’re nowhere we want to be – and we know we can do even better.

Oakland citywide on average is keeping pace with the state’s level of improvement in both ELA and Math. At this rate, however, it’ll take until 2040 for over 75% of students to be proficient in math and ELA.

 

The Painful Distribution of Proficiency

The slow progress is just an average across the whole city. Average schoolwide proficiency has a painful geographic distribution as well.

Last week, I attended an OUSD school board meeting where parents from The Oakland REACH turned out in full force to demand an “Opportunity Ticket” to get better access to quality schools. I can see where they’re coming from. I plotted schools’ SBAC proficiency on a map. And where I lived in Oakland when I was a kid, it’s surrounded by red dots – public schools where fewer than 35% of students are proficient.

 

SBAC proficiency rates of schools mapped across Oakland. Color corresponds from dark red to dark green, reflecting range from 0-100% proficiency rate on SBAC math or ELA test, respectively. Click on the photo to view the dashboard.

 

 

 

 

 

 

 

 

 

I’m glad that our city is improving on the whole (slow progress, but nevertheless upward progress. Wooooo!), and I also worry the most about the students in the city who need support and resources most urgently. Oakland Achieves recently conducted an analysis of African American students in 3rd–8th grade and how they did in on the CA School Dashboard (figures below). It’s truly shocking how few African American students are in schools in which students are performing above standard and/or improving. Particularly striking is how African American students are overrepresented in the category “Schools to Review,” which are schools that are scoring low on the SBAC tests and not closing the proficiency gap from year to year (aka falling more behind).

 

Screenshots from Oakland Achieves report on CA School Dashboard. Comparison of distribution of all students in Oakland vs. focus group on African American students.

 

What’s next:

So, I wrote this whole post on SBAC results…and SBAC proficiency is only ONE indicator of a school and student’s progress (albeit a very ­­important indicator). Our schools and students are so much more than just a score on a test (though positive progress is always a good sign and performance gaps amongst historically underserved subgroups are never a good sign). We and the state recognize this, hence the multimeasure approach of the CA School Dashboard, which combines a couple of key academic and culture/climate indicator. Good news: CA Dashboard data is out! And the Alameda County Office of Education, which is playing an ever more important role in Oakland public education, has a page dedicated to it with good resources, along with some additional dashboards from OUSD.

Keep an eye out for a post in 2019 as we dig into the results.

Happy end of 2018 everyone! I’m excited to dig into the data with y’all in the year ahead!

The Importance of Attendance…For Everyone!

As Superintendent Kyla Johnson-Trammell included in her back-to-school welcome message, attendance is super important. Inspired by her call for building positive attendance habits early in our young learners and since September was Attendance Awareness Month, I decided to dive deeper into the topic, specifically chronic absence, to help give context on why it matters so much – for everyone!

What is Chronic Absence?

Chronic absence is another way of looking at attendance by focusing on proportion of kids who missed a significant amount of school days, those who are chronically absent. For the first time, the state has defined how it will collect and calculate chronic absenteeism (see footnote for definition). This data is one of several indicators in the California School Dashboard that attempts to reflect school culture and climate and build toward a more holistic view of school performance.

In 2016-17 (the first year data was collected), Oakland’s citywide overall chronic absenteeism rate was 15.4% of students – that’s over 1 in 7 children across district and charter schools who miss 10% or more of their school days. For comparison, the statewide average is 10.8% of students in 2016-17 and recent analysis by AttendanceWorks puts the nationwide rate at 15.5% during 2015-16 (though they define chronic absence slightly different).

Why it matters:

  • For System Leaders: Because a lot of school funding in California is based on “average daily attendance” (ADA), not enrollment, a school loses approximately $85 per day for every student absent. This can add up quickly! For a district or charter with scarce resources and tight budgets (aka almost every school in Oakland), this feels important. To get an estimate of the potential minimum dollars lost, I did a quick back-of-the-envelope calculation. (Assuming 10% of school year = 18 days, 15.4% of Oakland’s 53089 students were chronically absent during attendance tracking period, and $15,337 per pupil funding annually in CA in 2016-17). That adds up to a potential loss of at least $12.6 million citywide! (Some students were absent a lot more than 18 days a year.)

18 days x(15.4% students chronically absent x 53089 students) x $15337 per pupil/180 days per year = $12.6M

  • For Students: Students who are behind and chronically absent are missing valuable instruction time – time that is necessary for closing the proficiency gap. When there are huge differences between subgroups, it becomes equitable access issue. To estimate the impact for socioeconomically disadvantaged students (SED), I made assumptions that: the 41.6K SED students citywide receive 6.5 instructional hours per day, attend school 180 days a year, chronically absent students miss at least 18 days (10%) of year, and used the citywide rate of 17.4%. That’s 849K total instructional hours lost in 2016-17. Every 1% citywide reduction in % of SED students who are chronically absent yields 49K extra instructional hours to work on closing the gap! Imagine all the reading or math that can happen in that time!

Chronic absenteeism rates citywide in Oakland by subgroup (CDE 2016-17)

  • For School Leaders: Attendance and chronic absenteeism rates can be leading indicators for a healthy school culture. Chronic absence is an early warning sign of academic distress, including school dropout (Oakland Achieves 2016). Unusual rates are symptomatic of other underlying issues and thus should be monitored carefully, acting as bellwethers for a healthy school culture. Some schools like Frick, MLA, and Hoover have been steadily reducing their chronic absence rates, showing consistent improvement not only in the past three years but also historically. They’re also schools with targeted and sustained climate and culture efforts from the school leaders.

Areas of Caution:

  • Focus on chronic absence rates beyond just the numbers. There are so many the factors that contribute to students’ chronic absenteeism, a variety inside and out of a family’s control. Health, transportation, family trauma, reactions to the national political climate – all this and more in Oakland. It’s necessary to understand the “whys” behind high rates and know each school site is different.
  • Metric not designed to capture change for individual students: Even if a school helps a student reduce the number of missed school days from 60 days annually down to 20, the student would still be considered chronically absent in both years because they’ve crossed the 10% threshold.
  • Differing definitions: The California Department of Education definition for chronic absence differs slightly than the federal definition (10% of school days vs. 15 days per school year). Chronic absence is calculated differently at continuation and alternative schools.

Snapshot of OUSD on the interactive map from The Hamilton Project, which displays federally collected chronic absence rates from 2015-16 on national, state, district, and school levels using the federal definition of 15+ days for chronic absence.

  • Reporting errors in first year: Not surprisingly, the first year of data collection for chronic absenteeism rates had its hiccups and inaccuracies as folks acclimated. Sites with flagged inaccuracies will have a banner to note this. See OUSD’s page for chronic absenteeism rates for example. (As a point of comparison: if you look at San Francisco’s, it’s much more wildly inaccurate.) Hopefully, data for 2017-18 will be more robust and accurate now that schools and districts have had time to adjust with first round of collection under their belts.

Looking forward:

I’m super excited (not surprisingly) about the possibility of updated 2017-18 chronic absence data in CA School Dashboard refresh (update scheduled late winter/early spring). In theory, chronic absence rates could be leading indicators, a bellwether to making sure students are getting the instructional hours they need, and we would have insight into how Oakland schools are progressing compared to statewide benchmark.

For those interested in learning more about chronic absence and ways to address it, I recommend checking out AttendanceWorks, a nonprofit dedicated to reducing chronic absence to advance student success, or OUSD’s Attendance webpage. Data geeks will appreciate the interactive map recently released by AttendanceWorks and The Hamilton Project at Brookings Institution on 2015-16 federal chronic absence data, allowing users to dig deeper into subgroup trends.

Footnotes: Definition: The California Ed Code defines a student as a chronic absentee if ”a pupil who is absent on 10 percent or more of the school days in the school year when the total number of days a pupil is absent is divided by the total number of days the pupil is enrolled and school was actually taught in the regular day schools of the district, exclusive of Saturdays and Sundays.”

Stopping the School-to-Prison Pipeline

According to the latest data, racial disparities in school discipline persist across the country. In April, the Government Accountability Office (GAO) released a report on national discipline trends and the Office on Civil Rights released its latest biennial data collection on school climate and safety. Both confirm that African Americans, boys, and students with disabilities are significantly overrepresented when it comes to being disciplined via suspension, expulsion, and other means.

Suspension rates alone are an inadequate indicator of school climate and mask the full extent of the problem. Some schools artificially lower their numbers by not reporting in-school suspensions. Also, schools can take other disciplinary actions on-site, formal and informal (e.g., office referrals) that can negatively affect students. So this data, which is self-reported by schools and districts, is only a part of the picture.

Why talk about school discipline?

  • Students of color are affected in every setting. Rich or poor, charter or district – it doesn’t matter. One of the key findings of the GAO report was that disparities in discipline rates between races persisted across income levels, type of public school attended, and type of disciplinary action.
  • Reducing suspensions matters. A lot. We need to reduce unnecessary suspensions as a step towards stopping school-to-prison pipelines. Some students, particularly those with disabilities or high-need backgrounds, who experience out-of-school suspensions and expulsions may benefit more from additional educational and counseling services (i.e. restorative justice circles) rather than isolation and exclusion.
  • It’s a civil rights issue. When students of color and those with disabilities are disproportionately suspended and expelled (sometimes for the same infractions as white or affluent students), it’s a sign that implicit biases may be affecting discipline decisions.

How is Oakland doing?

Many of our schools have been purposefully changing their discipline policies and implementing restorative justice and other similar practices. In the spirit of celebrating those locally who made strides in reducing suspension rates (shout-out to Oakland for beating the national rate!), let’s look at some highlights.

  • Some progress in most recent data despite citywide status quo: The average citywide suspension rate remained the same from 2015-16 to 2016-17 at 4.2%. Below are some schools across Oakland who reduced their suspension rates in a year with no citywide trend change.

  • Trend of improvement across the board for past six years: Compared to district rates in 2011-12 (the last year that’s conveniently available on Dataquest), there has been a decrease in suspension rates across all racial groups (see graph). I’m personally excited to see all those lines with downward trends (aka decreasing suspension rates, which is a good thing); for me, it means that school leaders have been focused on ending the school-to-prison pipeline in Oakland over the past few years. OUSD deserves applause for its vanguard work and widespread adoption of restorative justice, which involves shifting mindsets as well as changing disciplinary practices.

  • Closing the gap for African American students: The line above with the sharpest decrease is for African American students – which is great news given how high the rate was back in 2011-12 (13.8%). Schools across Oakland have reduced the suspension rate of African American students by over 5.2 percentage points, surpassing the average 3.4 percentage point decrease – necessary to eventually eliminate disproportionality.

We must continue this progress!

  • Oakland is still higher than the state average: From 2011-12 to 2016-17 (coincidentally the years the Obama administration issued guidance for disproportionate rates of discipline), Oakland improved at quicker rate than the state at reducing suspension rates (Oakland: 7.6% to 4.2%; State: 5.7% to 3.6%). However, our citywide suspension rate still remains higher than statewide (4.2% vs 3.6%). To put into context, 4.2% of students translate to approximately 2,300 Oakland students who received at least one suspension in 2016-17.
  • And we see disparities like those in state and national data: During 2016-17, African American students (8.6%), students with disabilities (9.2%), homeless students (8.8%), and foster students (12.2%) were more than twice as likely to be suspended than the average student (4.2%). These trends reflect statewide disparities in suspension rates. Groups nationally are digging deeper into causes behind the disparities to identify opportunity areas for improvement.
  • So we must pick up the pace to continue historical progress: From 2015-16 to 2016-17, most subgroups in Oakland did not see a significant decrease (>2%) in average suspension rates, with notable exceptions for foster and homeless students. Some subgroups saw increases in suspension rates from 15-16 to 16-17, like students with disabilities (9.2% in 16-17, +0.6% increase).
  • Especially since it is now part of the State Dashboard: Last spring, the State decided to incorporate suspension rates into its accountability system, the new CA School Dashboard. This multi-measure approach adds an indicator of school climate in addition to measuring academic performance via test scores. Theoretically, this will make it easier for parents to consider this aspect of their students’ experience, creating further pressure for districts and schools to improve.

Definitions:
Suspension rate = Unduplicated Count of Students Suspended divided by the Cumulative Enrollment at a single site for an academic year

Data sources:
CA School Dashboard data for Suspension, Fall 2017 update
CDE Dataquest
Civil Rights Data Collection (CRDC) for the 2015-16 School Year

Growth superstars – schools on the move!

If we looked at which schools had the highest absolute performance on statewide standardized tests in 2016-17, we would find that many schools follow the socioeconomic trends across Oakland. Some notable exceptions: Downtown Charter, Oakland Charter High, and Lighthouse High, who serve students in the Fruitvale/San Antonio/East Oakland serving mostly FRL students and getting excellent results.

Figure 1. Oakland public schools with 70% or higher proficiency as measured by SBAC in 2016-17.
Dark Blue = Both ELA and Math. Light Blue = Either ELA or Math only.

Focusing just on proficiency can be a misleading sole indicator of academic excellence, especially considering many Oakland students enter the classroom far behind grade level. It’s important to analyze how well a school helps their students grow and/or “beat the odds.”

So I created a set of criteria using SBAC scores in different ways to sift out “hidden gems:” schools who might not have super-high absolute proficiency rates (yet) but are doing a great job particularly for the most vulnerable students.

Criteria for “knocking it out of the park” on growing their students and beating the odds:

  • Blue level growth on average on CA Dashboard. This translates to 91+ growth percentile statewide, a comparison set that ranges from Piedmont to LA schools.
  • CCSA SSM ranking of 7+. Students’ proficiency rates are above the 70th percentile compared to California schools with similar demographics. This analysis replicates the approach used by the state of CA several years ago.
  • No huge inequities exist. Schools must be growing their different subgroups, particularly those historically underserved, at similar rates to be equitable. To receive an “None” for inequities rating, a school must be closing the proficiency gap for all numerically significant subgroups.
  • Growth in both English Language Arts and Math. Schools must be growing students in both categories.

Caveats:

  • Grades 3-8 only: It’s hard to gauge growth for high school students using SBAC because only 11th grade is tested, and comparing this year’s 11th graders to last year’s 11th graders doesn’t accurately reflect student growth.
  • Subgroup data availability: To qualify as a publicly trackable subgroup at the school, there must be more than 10 students in the subgroup in order protect student privacy in public data. Most schools have some subgroups based on race and income level. Many schools also have subgroups for English Language Learners, Students with Special Needs, and Foster Children.

After this list of crazy hard criteria, 3 schools meet these requirements: (in alphabetical order)

  • Aspire Golden State (6-12)
  • Aspire Monarch (TK-5)
  • Coliseum College Prep Academy (6-12)

Runner ups:

  • Aspire Lionel Wilson (6-12)
  • Aspire Triumph Tech (TK-5)
  • Bay Area Technology (6-12)
  • MPA Lower (TK-5)
  • Oakland Charter Academy (6-8)
  • Thornhill Elementary (K-5)

Notable mentions:

  • For no inequities in one of the two academic categories: ASCEND (K-8), EnCompass (K-5), Esperanza (K-5), MPA Upper (6-12)
  • For particularly growing African-Americans: Aspire Golden State, BayTech, CCPA, Encompass (ELA only), MPA Lower (Math only), Thornhill (ELA only)

Though the first rounds of enrollment has passed, I still encourage families to dig deeper into the data behind their school, especially for families of color, to better inform and impact their future school’s decision making body. (FYI, if you haven’t submitted an enrollment application already, second round applications are available). A fellow data nerd has looked at some schools in Oakland that are getting great results for students of color. Hopefully this blog post helps us celebrate how fast some of our students and schools are growing!

How did Oakland do on the latest state tests?

The latest state test scores came out recently. Those of us education acronym lovers call it the “SBAC,” which stands for the Smarter Balanced Assessment Consortium which a multi-state public agency that developed the new, harder, common core-aligned tests. The recent results were from grade 3-8 and grade 11 students who took the test about 6 months ago, in Spring 2017. There aren’t many statewide trends beyond stagnation and persistent achievement gaps. What about Oakland? Here I offer three fast facts, plus three pieces of good news, three bad, and one important caveat.

First, Fast Facts:

  • Citywide, about one-third of students meet or exceed proficiency standards in math and ELA (specifically, 35% in ELA and 28% in Math.) This includes data from all district-run and charter public schools in Oakland.
  • Statewide, 48% of students were proficient in ELA and 38% of students were proficient in Math.
  • Notably, 3rd and 4th graders performed higher on math than other grades, which make sense given that they’re the students who were exposed to Common Core at an early age – promising results for the future!

The Good News:

  • In 13 Oakland schools, 70% of students met or exceed standards in Math, ELA or both – resulting in a top rating (Blue or Tier 5) on SBAC performance. The schools below are listed in order of greatest total differential—who had the highest percentage of students proficient and above in ELA and math, combined, compared to the Oakland school averages (ELA average = 35%, Math = 28%). Special shout-outs for ARISE High who made huge leaps this year (+22% ELA, +15% Math), as did Francophone (+6% ELA, +23% Math)!
  • Slow but steady improvement amongst vulnerable subgroups. Across all Oakland schools citywide, vulnerable subgroups (students with disabilities, English learners, socioeconomically disadvantaged, African-American, Latino) showed gains (although too minor to be considered gap-closing, imho) on average across Oakland schools (0.4% to 1.4% proficiency increase).

    Click on the map to see the top and bottom performing Oakland public schools on the 16-17 SBAC.

  • Oakland is beating the statewide average in growth of % of students proficient/advanced! Though there has been little growth statewide (±0.5%), Oakland experienced minor improvement citywide from the prior year’s scores; the percent of Oakland students who met or exceeded standards in both ELA and Math grew by 3 percentage points from ’15-16.

The Bad News:

  • Compared to the state, Oakland is way overrepresented among the bottom 5% statewide. Oakland has 26 schools in the bottom 5% statewide (representing 21% of 124 Oakland public schools with scores), more than any city other than Los Angeles, which has 31 – which has 10x the number of students compared to Oakland – see this LA Times analysis). This is a big deal because under federal law (ESSA), these are the schools that “have to” be improved.
  • The schools in the bottom 5% of absolute performance generally saw little growth. Of the bottom 15 schools in ELA and Math in 2016 (excluding alternative schools), the new results show little movement (modest gains/losses ±3% for most schools, which would be considered “maintaining” or no improvement) in Math. Only two showed significant growth in ELA. Here is a map of these schools, which are primarily in West and East Oakland.
  • Even for a large urban district, Oakland is in bottom third. Oakland is the 12th largest district in California (with approx. 73% low-income students). With 35% proficiency in ELA and 28% in Math, Oakland is in the bottom third compared to the top 10 largest districts in both Math and ELA (though Oakland improved slightly more than others).

The Caveat:

This is just a preliminary analysis of an imperfect measure. Using SBAC is imperfect in many ways – a long list that many have already opined on. An especially important imperfection from my perspective is that the state does not use the growth of the individual students. They compare this year’s 6th grade class with last year’s 6th grade class, which does not control for factors like incoming student level. We need a different way to gauge individual student growth and determine if it is on par statewide or worsening through the years.

This latest release of SBAC scores scratches the surface. In an upcoming post, we’ll look more closely at DFM (a new acronym! “Distance From Met”) – one of the most powerful tools for measuring student growth in this latest state testing system.

And of course, ultimately, ELA and Math test scores are not enough. We’re excited that OUSD is developing a better way to communicate about the new CA dashboard which includes more measures for a more holistic view of schools (keep an eye out, it’s called Oakland Public School Report Cards!)

Stay tuned for more!

Looking back, looking forward, looking to you

Over the summer, I’ve been reflecting on the past year. Two decades ago, I was a student in Oakland Unified School District, and now I have the honor of analyzing data and writing about Oakland public schools through our CRUNCHED! blog we launched on November 21, 2016. I’ve written ten entries over the last eight months, and have tons of ideas for the year ahead, but thought it would be worth pausing to reflect, share some thoughts, and get your take on what data we should crunch in the year ahead.

I’ve realized I haven’t really written about why I’m writing CRUNCHED! For me, writing these blog posts is personal. This might sound odd given that I’m writing about data analyses. Let me explain. Over the past year, I have gravitated to topics that resonate with me based on my personal experiences in Oakland schools. I constantly stress the need for good, quality schools because I’ve attended ones that were unsafe and not rigorous. I focus on results for English Learners because I once was one and had the fortune of being in a now-defunct OUSD bilingual elementary program. I remember realizing I was one of the handful of free/reduced lunch students in AP Calculus in my high school, which is why I now constantly look at achievement gaps of under-represented subgroups.

Me being very geeky happy when I won the “Transparency” award at a team dinner.

Throughout the blog posts, I’ve tried my best to keep equity at the forefront (one of my and Educate78’s core values) by looking for SBAC bright spots to highlight Oakland schools that do amazing work for subgroups often left behind. As a young female professional in the data world, I’m incredibly sensitive to gender gaps, especially in pay and leadership positions (pause for a round of applause for our new Superintendent Kyla Johnson-Trammell for smashing that historical glass ceiling!).

Variety of graphics from past posts

Since I launched this blog less than a year ago in November, 2016, I’ve been excited to expand CRUNCHED! beyond test scores (like SBAC state testing) to include a variety of other important topics like teacher retention, the history of superintendents, and the accountability through the new CA data dashboard. They’re all critical to understand as they are all important factors that affect Oakland students’ education. I’ve learned a lot in the process of researching and writing about them.

The Oakland public education landscape when I was in elementary school is very different from today’s. Unfortunately, some things are the same: still not enough great schools for kids to attend (blue tier). We’ve come so far, but we have so, so much more work to do!

I hope that, through crunching data and sharing the findings, I can play a small role in informing all the folks who are working to improve public education, especially for the kids who were like me not too long ago. To better inform, I’d love to hear what data you all think would be helpful in this coming year!

Please email me and let me know what data questions are keeping you up at night – we’ll add to the list as it grows. Here are some topics we’re considering (and when):

  • Deeper dive into the data from the recent #InformingEquity ERS study (maybe the first part of the school year)
  • SBAC results of course, overall, bright spots, etc. (starting in September)
  • Fall enrollment numbers (probably in early October)
  • Update of the state dashboard (looking at December)
  • Release of the SRA (probably in spring 2018)
  • Graduation rates (as we approach the end of the school year)

School hasn’t started yet, but I’m already pumped for the 2017-18 school year! Help me keep crunching by reading, sharing this blog with others (using the buttons below!), sharing your feedback with me, and putting the data to work to help students, whatever your role in our city!

Share this post!