How did Oakland do on the latest state tests?

The latest state test scores came out recently. Those of us education acronym lovers call it the “SBAC,” which stands for the Smarter Balanced Assessment Consortium which a multi-state public agency that developed the new, harder, common core-aligned tests. The recent results were from grade 3-8 and grade 11 students who took the test about 6 months ago, in Spring 2017. There aren’t many statewide trends beyond stagnation and persistent achievement gaps. What about Oakland? Here I offer three fast facts, plus three pieces of good news, three bad, and one important caveat.

First, Fast Facts:

  • Citywide, about one-third of students meet or exceed proficiency standards in math and ELA (specifically, 35% in ELA and 28% in Math.) This includes data from all district-run and charter public schools in Oakland.
  • Statewide, 48% of students were proficient in ELA and 38% of students were proficient in Math.
  • Notably, 3rd and 4th graders performed higher on math than other grades, which make sense given that they’re the students who were exposed to Common Core at an early age – promising results for the future!

The Good News:

  • In 13 Oakland schools, 70% of students met or exceed standards in Math, ELA or both – resulting in a top rating (Blue or Tier 5) on SBAC performance. The schools below are listed in order of greatest total differential—who had the highest percentage of students proficient and above in ELA and math, combined, compared to the Oakland school averages (ELA average = 35%, Math = 28%). Special shout-outs for ARISE High who made huge leaps this year (+22% ELA, +15% Math), as did Francophone (+6% ELA, +23% Math)!
  • Slow but steady improvement amongst vulnerable subgroups. Across all Oakland schools citywide, vulnerable subgroups (students with disabilities, English learners, socioeconomically disadvantaged, African-American, Latino) showed gains (although too minor to be considered gap-closing, imho) on average across Oakland schools (0.4% to 1.4% proficiency increase).

    Click on the map to see the top and bottom performing Oakland public schools on the 16-17 SBAC.

  • Oakland is beating the statewide average in growth of % of students proficient/advanced! Though there has been little growth statewide (±0.5%), Oakland experienced minor improvement citywide from the prior year’s scores; the percent of Oakland students who met or exceeded standards in both ELA and Math grew by 3 percentage points from ’15-16.

The Bad News:

  • Compared to the state, Oakland is way overrepresented among the bottom 5% statewide. Oakland has 26 schools in the bottom 5% statewide (representing 21% of 124 Oakland public schools with scores), more than any city other than Los Angeles, which has 31 – which has 10x the number of students compared to Oakland – see this LA Times analysis). This is a big deal because under federal law (ESSA), these are the schools that “have to” be improved.
  • The schools in the bottom 5% of absolute performance generally saw little growth. Of the bottom 15 schools in ELA and Math in 2016 (excluding alternative schools), the new results show little movement (modest gains/losses ±3% for most schools, which would be considered “maintaining” or no improvement) in Math. Only two showed significant growth in ELA. Here is a map of these schools, which are primarily in West and East Oakland.
  • Even for a large urban district, Oakland is in bottom third. Oakland is the 12th largest district in California (with approx. 73% low-income students). With 35% proficiency in ELA and 28% in Math, Oakland is in the bottom third compared to the top 10 largest districts in both Math and ELA (though Oakland improved slightly more than others).

The Caveat:

This is just a preliminary analysis of an imperfect measure. Using SBAC is imperfect in many ways – a long list that many have already opined on. An especially important imperfection from my perspective is that the state does not use the growth of the individual students. They compare this year’s 6th grade class with last year’s 6th grade class, which does not control for factors like incoming student level. We need a different way to gauge individual student growth and determine if it is on par statewide or worsening through the years.

This latest release of SBAC scores scratches the surface. In an upcoming post, we’ll look more closely at DFM (a new acronym! “Distance From Met”) – one of the most powerful tools for measuring student growth in this latest state testing system.

And of course, ultimately, ELA and Math test scores are not enough. We’re excited that OUSD is developing a better way to communicate about the new CA dashboard which includes more measures for a more holistic view of schools (keep an eye out, it’s called Oakland Public School Report Cards!)

Stay tuned for more!

Looking back, looking forward, looking to you

Over the summer, I’ve been reflecting on the past year. Two decades ago, I was a student in Oakland Unified School District, and now I have the honor of analyzing data and writing about Oakland public schools through our CRUNCHED! blog we launched on November 21, 2016. I’ve written ten entries over the last eight months, and have tons of ideas for the year ahead, but thought it would be worth pausing to reflect, share some thoughts, and get your take on what data we should crunch in the year ahead.

I’ve realized I haven’t really written about why I’m writing CRUNCHED! For me, writing these blog posts is personal. This might sound odd given that I’m writing about data analyses. Let me explain. Over the past year, I have gravitated to topics that resonate with me based on my personal experiences in Oakland schools. I constantly stress the need for good, quality schools because I’ve attended ones that were unsafe and not rigorous. I focus on results for English Learners because I once was one and had the fortune of being in a now-defunct OUSD bilingual elementary program. I remember realizing I was one of the handful of free/reduced lunch students in AP Calculus in my high school, which is why I now constantly look at achievement gaps of under-represented subgroups.

Me being very geeky happy when I won the “Transparency” award at a team dinner.

Throughout the blog posts, I’ve tried my best to keep equity at the forefront (one of my and Educate78’s core values) by looking for SBAC bright spots to highlight Oakland schools that do amazing work for subgroups often left behind. As a young female professional in the data world, I’m incredibly sensitive to gender gaps, especially in pay and leadership positions (pause for a round of applause for our new Superintendent Kyla Johnson-Trammell for smashing that historical glass ceiling!).

Variety of graphics from past posts

Since I launched this blog less than a year ago in November, 2016, I’ve been excited to expand CRUNCHED! beyond test scores (like SBAC state testing) to include a variety of other important topics like teacher retention, the history of superintendents, and the accountability through the new CA data dashboard. They’re all critical to understand as they are all important factors that affect Oakland students’ education. I’ve learned a lot in the process of researching and writing about them.

The Oakland public education landscape when I was in elementary school is very different from today’s. Unfortunately, some things are the same: still not enough great schools for kids to attend (blue tier). We’ve come so far, but we have so, so much more work to do!

I hope that, through crunching data and sharing the findings, I can play a small role in informing all the folks who are working to improve public education, especially for the kids who were like me not too long ago. To better inform, I’d love to hear what data you all think would be helpful in this coming year!

Please email me and let me know what data questions are keeping you up at night – we’ll add to the list as it grows. Here are some topics we’re considering (and when):

  • Deeper dive into the data from the recent #InformingEquity ERS study (maybe the first part of the school year)
  • SBAC results of course, overall, bright spots, etc. (starting in September)
  • Fall enrollment numbers (probably in early October)
  • Update of the state dashboard (looking at December)
  • Release of the SRA (probably in spring 2018)
  • Graduation rates (as we approach the end of the school year)

School hasn’t started yet, but I’m already pumped for the 2017-18 school year! Help me keep crunching by reading, sharing this blog with others (using the buttons below!), sharing your feedback with me, and putting the data to work to help students, whatever your role in our city!

Share this post!

Supt Glass Ceiling (Re)Shatters; How About Equal Pay?

I’m so excited about the appointment of a Kyla Johnson-Trammel as the next superintendent; she’s female and attended OUSD schools (like me), and in many ways reflects the community she’s serving. In one of my previous blog posts, we saw how few female superintendents there have been in Oakland over the pasts 50 years (7 of 24- just under 30%), whether selected by our local Board or state-appointed.

Kyla will re-shatter the glass ceiling- sort of: There hasn’t been a locally selected female Superintendent in Oakland in almost two decades, with the last one being Carole Quan (1997-99).

This issue hits really close to home for me as a young female working in a field (data) that is traditionally dominated by males. Here are some interesting points to consider:

The glass ceiling still exists, despite education being an otherwise female-dominated field. From our previous blog post on Superintendents of Oakland in past 50 years, we know that the glass ceiling still exists, and not just in Oakland. Less than one-third of the Superintendents in Alameda County are female in 2015, even though the majority of teachers are female (OUSD teachers = 71% female), making Johnson-Trammell’s appointment that much more important.

Yet women are still paid less on average. Common (and unfortunate) across many sectors: women tend to be paid less than their male counterparts for same position or work. This remains an issue in our own backyard. Across Alameda County, female Superintendents are paid, on average, $29,167 less than male Superintendents in base salary ($250,700 vs $221,533), partially because all larger districts are male-led (again, scoring the importance of Johnson-Trammell’s appointment for the largest school district in Alameda County).

How will Johnson-Trammell’s salary stack against previous male predecessors? Average for past OUSD head honchos (permanent and interim from 2012-15) is $294,776 in total pay ($275,911 in base salary, not adjusted for inflation, though the numbers are all pretty recent); remember, they all have been male. According to the Board report for this Wednesday’s meeting, Superintendent Johnson-Trammell will be paid slightly less than the average: $280,000 in base pay with a little over $20,000 in fringe benefits.

What is considered “fair” compensation for a Superintendent? (Note: we’ll look at total pay, which includes other pay packages and salary but exclude benefits for simplicity and fairness.) Again, some points to consider:

Based on district size: The size of the district matters: it’s easier to manage a district serving 1,000 students than 10,000. All other things being equal, superintendents in bigger districts should be paid more than in smaller districts. OUSD operates or oversees schools serving a total of about 50,000 students. Projecting from other local Superintendent salaries, OUSD’s Superintendent should be paid $323,924 based on size of district.

Based on student demographics: The student population matters because it affects the budget and other factors. Comparing Oakland with Livermore, which serves fewer and more affluent students, might not be a helpful benchmark. So, let’s offer a more “apples-to-apples” view and look at other similar school districts across California that have similar student demographics and size. On average, OUSD compensates Superintendents a little more than these similar districts by $24K in total pay on average ($294,776 vs $270,955).

Student enrollment vs total pay for Superintendents in Alameda County, CA in 2015. Green = male, Yellow = female.

Compensation is a lot more than just base pay: Base salary often receives the most attention, but some Superintendents receive substantial value ($62K!!) in “other pay” that contributes to their total pay. Some of these additional components can be compensating a Superintendent for a smaller base salary; it can also lead to egregious padding for the top guns. (For the curious: John Collins of Poway Unified had $62K in “other pay” in addition to the $309K base salary, bringing his total pay to $371K for a district enrollment of 35K students in 2015.) Benefits are important, too, as they contribute to how much a school district must shell out to employ one person. The range differs in orders of magnitude from $853 to $96,660.

Some might argue that there are other factors that should be considered, including familiar ones like credentials or teaching/managing experience, and some harder-to-evaluate ones like level of complexity. I couldn’t find reliable numbers on these, but if pay were higher for “challenging” situations, given our current budget situation, unsatisfactory school quality, and passionate political debates at school board meetings, Superintendent Johnson Trammell would need to be paid way more that we can afford!

Overall, it appears our new Superintendent will be compensated appropriately – higher than the “salary cap” that some in the community have suggested, but less than what her male predecessor made. We’ll stay tuned in to see exactly how things shake out at the June 14 Board meeting.

For my fellow data nerds, I’ve pulled together some salary info of our county’s Superintendents and superintendents from similar districts as a “starter pack. Salaries of many public employees, including superintendents, from 2011-2015 are available through Transparent California, a public group dedicated to sharing out accurate information on compensation of public employees.

###

My name is Carrie Chan, and I’m Educate78’s newest staff member. I joined the organization as an Analyst, and I LOVE data (feel free to call me a data geek). As a former OUSD student, I also care a lot about Oakland public schools. That is why I am so excited about this new blog series, “Crunched!” which will take a data-driven approach to important, relevant questions facing Oakland public schools. Please email me with ideas or requests.

Share this post!

YES, THE SRA IS OUT!

It’s the best time of the year for any Oakland data nerd. It’s when data starts raining from the district and state!

I was thrilled to see that OUSD has released the Strategic Regional Analysis (SRA) for 2016-17. This second edition of the SRA includes many data sets on OUSD schools across the five regions of the city (Central, East, Northeast, Northwest, West). Kudos to OUSD for this progress towards more transparency and to the Data & Assessment team for their impressive work. I’m excited to see how this data informs OUSD decisions.

For those of you who remember last year’s SRA, there are three key differences in this year’s:

  • Interactive data visualization: Instead of a printable ginormous report with static graphs like last year’s, this year’s SRA presents information in forms of interactive dashboards, enabling each user to examine the data for answers to his/her own questions. You can focus in on one school, compare regions, or choose to look at trends for the district as whole.
  • Less guided experience: Without the structure of a report narrative, users may miss interesting patterns and important conclusions from the data, particularly users who do not have a data background or comfort in navigating data visualizations. While the Executive Summary does include a few key takeaways, I’d love to see more. Later in this blog I’ll offer a few of mine.
  • Deeper dive: This SRA dives deeper than before in some topics; for example, Teacher Retention expanded into Teacher Retention and Experience Levels.

Key takeaway #1: It’s a myth that Oakland enrollment is drastically declining! I’ve heard lots of people say that OUSD has been hemorrhaging students over the past few years. But actually, the number of students in district-run schools has held pretty steady over the past five years, enrollment staying within ±1% change each year. There has actually been a 11% growth in public school enrollment over the past five years, but charter schools are capturing the majority of that growth, including a 29% bump from 2014-2015 to 2015-2016

Snapshot from SRA Dashboard on enrollment. *including ACOE and OUSD charters

Question to Consider: if OUSD district enrollment has remained steady in past few years and per pupil revenue has been growing, how have we found ourselves in this current budget crisis? (We’ll explore this question in a new series on the OUSD budget.)

Key takeaway #2: We need more quality schools! (I feel like a broken record harping on and on about school quality, but don’t we all want more good schools for our students?!)  The number of quality schools (green or blue on the SPF tiering) drop as we move up grades. 10 of 54 Elementary schools and 3 of 22 Middle schools are blue or green. There are 2 “green tier” High schools – not a single top-tier district-run high school in Oakland. That translates to less than 1 in 5 students who have access to a quality district school in Oakland.

Snapshot from OUSD SRA dashboard on SPF, showing overall SPF tier for high schools only.

 

Key Takeaway #3: Huge variation in attrition rates by region. The rate at which OUSD students stay in public schools (retention) as they matriculate to middle and high school varies widely by region. Maybe this isn’t big news to folks – but I thought it was interesting that in the more affluent Northwest region, nearly one-third of OUSD 5th graders leave the Oakland public school system for 6th grade! Compare that to only 8% in the East region. The same happens again, where another one-third of OUSD 8th graders in the Northwest region leaves the system (compare 10% in East region).

Snapshot from SRA dashboard on Attrition and Transition for 5th to 6th grade.

What are your key takeaways? Send me an email with your insight and a screenshot of the relevant data visualization and I’ll add them to a future post as I continue to dive into the SRA. #datanerdsunite

More to come—stay tuned!

###

My name is Carrie Chan, and I’m Educate78’s newest staff member. I joined the organization as an Analyst, and I LOVE data (feel free to call me a data geek). As a former OUSD student, I also care a lot about Oakland public schools. That is why I am so excited about this new blog series, “Crunched!” which will take a data-driven approach to important, relevant questions facing Oakland public schools. Please email me with ideas or requests.

Share this post!

The Old and The New: Comparing the Academic Performance Index and New California School Dashboard

Two weeks ago, the state unveiled publicly the first iteration of the new CA School Dashboard, providing student performance data and finally replacing the old Academic Performance Index (API), which was last issued for 2012-2013. Data geeks rejoice!

The Good, Bad, and Ugly

The new CA Dashboard is multi-measure and includes not only academic indicators (i.e. SBAC, grad rates) but also indicators of student climate (i.e. suspension rates). That’s useful for providing a more comprehensive picture of schools. Happily, the new CA system also considers growth, not just absolute performance (take note, Betsy DeVos!). Importantly, it also makes subgroup performance more visible – critical to understand if a school is helping to close the achievement gap for historically underserved populations.

Comparison of Academic Performance Index (API) and New CA Dashboard

These are all important improvements. But it’s not user-friendly. Unlike the old API, which boiled down to a single number plus two decile rankings, there is no single summary score and no way to easily compare against other schools in the dashboard. For each indicator, you need to look at the additional “Five by Five Placement Grid Reports” to understand why the school received the color coding it did. Some of the data currently in the dashboard is old while other data points are missing. And the Dashboard is only offered in English and is not sortable / printable / downloadable. Talk about jumping through hurdles for anyone (particularly families) who want to use this tool to figure out how a school is doing.

*Although the CDE website doesn’t allow comparisons, EdSource published an awesome tool to enable easy comparison of schools and districts on each indicator.

How are Oakland schools doing?

With the addition of all climate metrics plus growth, I wondered how Oakland schools are rated in the new system compared to our rankings under the old API system. I had to make some assumptions: I converted the API decile rankings into 5 colored tiers and took a straight average of all the metrics to come up with single summary color for each school in the new system.**

** API rankings to color conversion: Deciles 1-2 = red, 3-4 = orange, 5-6 = yellow, 7-8 = green, 9-10 = blue. New Dashboard indicators to summary color conversion: I converted the color for each indicator into a numerical value (red=1; blue=5), calculated a simple average across all the indicators available for each school, and translated that average back into the corresponding color. Note that many indicators (such as college readiness) were not available in this first roll-out, and school tiers might change with the inclusion of this data in the fall.

Some observations:

  • A different distribution: Under the old system, Oakland has a depressing number and proportion of schools that were in the lowest two deciles – a left-tilting ski slope. Under this new system, we seem to have fewer terrible schools, more mediocre and fair schools, and fewer excellent schools – more like a bell curve. This is not surprising since the cutpoints for the new system were designed to create a curve.
  • Majority still unacceptable: Even with the new system scores across multiple measures, almost 60% of Oakland schools are still in the bottom two tiers (i.e. “red” and “orange” schools) – not that different from the API-era rankings.
  • Excellence in short supply: In the new system, some schools that historically scored well on the API don’t do quite as well – due to lack of growth, poor subgroup performance, or lower climate scores.

The bottom line: we have a lot of work to do to create the school system that our kids deserve!

Next time: The Power of the 5x5 and Did schools change their colors?

###

My name is Carrie Chan, and I’m Educate78’s newest staff member. I joined the organization as an Analyst, and I LOVE data (feel free to call me a data geek). As a former OUSD student, I also care a lot about Oakland public schools. That is why I am so excited about this new blog series, “Crunched!” which will take a data-driven approach to important, relevant questions facing Oakland public schools. Please email me with ideas or requests.

Share this post!

“Keep Our Oakland Teachers!” – Teacher Retention

The data gurus at OUSD recently released a dashboard focused on teacher retention rates in OUSD. (Side note: I’ve found this announcement page from OUSD Data to be very helpful in keeping track of their newest data releases and news!) I’ve been thinking a lot about teacher retention ever since our Celebrate Oakland Teachers Night (plus all those news reports about California’s dire teacher shortage) – so of course I had to dig deeper!

Question 1: How high (or low) is teacher retention in Oakland public schools?

Answer: Three-year retention ranges from 40% to 70%, varying school and region.

Specifically, middle schools and the schools in the Elevation network (which are the schools that the district is making an intensive effort to improve) have the highest turnover. On average, a student in an OUSD middle school will watch two-thirds of the teachers leave by the time s/he completes 8th grade. Yikes! It raises an important question: How, as a city and a community, can we ensure that our students with the greatest needs get our city’s best educators?

Figure 1. Average teacher retention rate at sites by Network. Courtesy of OUSD.
Schools in the Northwest have higher teacher retention than schools in the East and West – not coincidentally the areas that have higher poverty and crime.
Figure 2. Average teacher retention rate at sites by SRA region. Courtesy of OUSD

A few other things to notice in this data:

  • One-year retention clearly doesn’t tell the whole picture: On an annual basis, teacher retention rates in some regions don’t look so bad (>80%). But, an average three-year site-based retention rate of 54%* district-wide is definitely not good – half of the teachers are different in three years!
  • Retention rates have declined slightly over the past 5 years: In the recent years (’11-12 to ’15-16), teacher retention rate in OUSD has dropped a little more each year from 83% to 79.5%. The drop in retention rates from 2008 to 2011 was almost certainly driven by recession-era reductions in available teaching positions.
Figure 3. Retention of teachers in OUSD from 2006-2016. Courtesy of OUSD

Eagle-eyed folks will notice that this data is limited to district-run schools. Unfortunately, teacher retention data for charter schools isn’t collected centrally or comprehensively. This is a gap I’d love to see addressed moving forward to provide a citywide look at retention!

Question 2: Are teacher retention rates and school quality correlated?

Teachers are the people directly working with our kids every day in the classroom. We often hear that teacher longevity directly impacts the quality of students’ educational experience. As a data nerd, of course I need to ask: What does the data say?

Answer: Yup!

There does seem to be a relationship, with higher teacher retention rates at schools with a higher School Performance Framework (SPF) score. While there is wide variation in Tier 2 and 3 schools, every single school in Tier 4 or 5 has an average retention rate of higher than 70%.

But, here is a big data nerd caution: even though there seems to be a correlation, we don’t know about causation! This may be a “chicken or egg” question: does high teacher turnover drive low student achievement and school culture? Or are working conditions and environment at the school accounting for the correlation between high turnover and high student need? Or, are both school quality and teacher retention symptoms of other causal factors?

A musing: You know what data I’d REALLY like know? Retention not schoolwide or region-wide or district-wide, but retention of our most impactful teachers – i.e. those teachers that are instructional wizards, amazing with family relationships, or really hold down a school culture. I wonder if the new Teacher Growth and Development System (TGDS) would provide data on this…

Questions 3: What can we do about this?

Answer: Glad you asked!

While we know about factors that increase teacher retention from a national perspective (see TNTP’s Greenhouse Schools report), we don’t have enough data locally to really understand the root causes of what’s causing our teachers to leave Oakland classrooms. (Of course I’m biased because I always think data is critical to getting to the answers!) To understand what affects our teachers’ engagement and satisfaction, Educate78’s Teacher Advisory Group is leading a citywide teacher retention survey. The results of this survey will enable us to connect local results with the national patterns that Gallup has seen: i.e. the more engaged a teacher is, the more likely they will stay in their current job.

PLEASE HELP US SHED LIGHT ON THIS TOPIC: ENCOURAGE ALL OAKLAND TEACHERS YOU KNOW TO TAKE THE SURVEY!

(PS: check out here what we did with this data last year! If you’re interested in learning more, keepan eye out for TeachOakland Grant page)

*This an average of the different regions. I recognized that averaging across the regions is not the best approach mathematically since the regions all differ in the number of teachers compared to the pure district-wide average, but the data is not downloadable for me to calculate quickly.

###

My name is Carrie Chan, and I’m Educate78’s newest staff member. I joined the organization as an Analyst, and I LOVE data (feel free to call me a data geek). As a former OUSD student, I also care a lot about Oakland public schools. That is why I am so excited about this new blog series, “Crunched!” which will take a data-driven approach to important, relevant questions facing Oakland public schools. Please email me with ideas or requests.

Share this post!