CRUNCHED: Are Oakland Graduates Prepared for College and Career?

 

 

In the last CRUNCHED! blog post, I looked at citywide graduation rates (takeaway: steadily improving year-over-year in Oakland, but gaps persist). As our recent grads start thinking about post-grad life, I wondered from a data perspective how to dig deeper into the question around whether students are prepared for life after high school. (Being “ready” can mean different things to different people; I’m not sure I felt fully prepared, or maybe no one ever really does.) But using what limited data is available, I asked the question: Are Oakland public high school grads getting the coursework that, according to the State of California, means they’re really academically ready for college and future careers?

 

Are Oakland’s students ready for success post-high school?

The goal of high schools should not only be to help their students graduate, but also to ensure that they’re ready for the post-high school world ahead of them. Focusing on 4-year cohort graduation rate as the sole measure of progress is not enough; a diploma alone no longer equals success in this modern post-high school world . One longtime proxy measure of this readiness has been to look at how many students are A-G ready, which means they’ve completed coursework that meets basic admission requirements for UC/CSUs, our two state university systems. By this traditional measure, almost half of the students in Oakland are prepared for college because they’ve taken and passed the coursework leading up to it.

Figure 1. Graph showing Oakland citywide (including charter and district high schools) 4-year cohort graduation outcomes for 2017-18.

Acknowledging that not all post-high school paths lead to 4-year colleges and that coursework alone is not enough to be prepared, the California Department of Education (CDE) included this past fall a College/Career Indicator (CCI), which has multiple pathways to gauge academic preparedness for post-high school life. Some pathways are more aligned for college and others more aligned for a career. Innovate Public Schools published an awesome, parent-friendly explainer on CCI that I highly recommend checking out.

Figure 2. Bar graph comparing different subgroup rates of 4-year cohort college/career readiness as defined by College/Career Preparedness Indicator from CA School Dashboard

Under this new metric, which now has a broader set of criteria to define preparedness, only 38% of the 4-year cohort are considered prepared for college/career post-grad pathways. 18% are approaching the level that would consider them prepared, while 42% are not prepared for college and/or career. The differences vary sharply across ethnicities, with Latinx and Black students (the two largest racial groups in Oakland) facing wildly different odds than Asian and White students. The data finds that an Asian student is more than 2X more likely to be prepared for college/career than a Black student.

Two reasons I’m hopeful:

1. Local bright spots. Focusing on how individual Oakland high schools prepare grads of color for post-HS life, there are schools with positive progress. 64% of COVA High’s African American students and 95% of CCPA’s majority Latinx grads (the highest percentage by far in the entire city!!!) were considered prepared on the Dashboard College/Career indicator in 2018.

2. A village coming together to support our kids. We hear stories of students not adjusting to college well, and those adjustments are hardest to make for those who are low-income and first in their family to pursue higher education. Anecdotally, I’ll offer my own experience: I was one of those students and who was NOT prepared for the socioemotional challenges in college and struggled throughout those years, especially with balancing familial obligations that so many low-income students shoulder starting early in their lives. It excites me to see a mighty “village” coming together to support our Oakland grads. Organization like Oakland Promise are scaling up their financial assistance programs (through financial aid navigation support in their future centers and $8.5M in scholarships) to help lower barriers to get into college, while organizations like East Bay College Fund and Beyond12 are going beyond K-12 by mentoring and supporting students while they’re in college.

So what’s next?

Even if a student gets into college or a career path, there’s currently no way to track whether they’re persisting and completing post high school programs or not. Currently, the general public doesn’t have much insight into the future of students after high school. The data nerd in me would love to see implementation of a longitudinal data system to track which students are persisting and actually succeeding, matching those figures with the numbers predicted by the College/Career Indicator. (Fingers crossed for bills for longitudinal data systems currently making their way through legislature!)

At the state level, the CDE has signaled clearly that simply graduating is not enough to be prepared in the 21st century. Though Oakland has improved overall graduations rates over the past few years, progress still needs to be made as only a little over half of the city’s graduates are ready for college and the work world—especially if we are to do right by our historically underserved students and ending cycles of lower educational preparedness.

CRUNCHED: Let’s Look at Oakland Graduation Rates

 

 

Around this time every year, I eagerly start following stories about graduation, including graduation rates (data nerd, can’t help myself). I hear about local schools like Lighthouse and McClymonds who are simply knocking it out of the park and helping students defy a too common historical narrative.

Before we start, here’s a primer into the key metric I’ll be looking at: 4-year cohort graduation rates. Imagine tracking all the 9th graders who enter an Oakland high school – what percentage of them will graduate from 12th grade in four years? That’s essentially how 4-year cohort graduation rates are calculated. (This adjusts for variables like subtracting students who transfer out and adding students who join the cohort later in 10th-12th grade.)

Why this new metric matters so much: 4-year cohort graduation rates look at how effective a school/district is at getting their students – regardless of what level they enter high school at – out the school doors, with a diploma, and hopefully ready for college and career (more on this last one in a following blog).

 

Two Reasons to be Hopeful:

1. Single-site Success Stories: There are some consistently successful high schools (e.g. Life, OSA) and high schools with strong recent progress (e.g. MPA Upper, Arise), but I wanted to highlight McClymonds for their impressive growth. To put into context the amazing feat that the Mack seniors and the supporting community (because it takes a village to raise our children) achieved last June, take a look below at McClymonds’s graduation rates over the past few years. In the space of 7 years, they went from a little over half of students graduating on time to nearly 8 out of 10 students graduating on time, with the dropout rate decreasing by 3 times. Wow!

Figure 1. Cohort Graduation and Dropout rates for McClymonds HS from 2011-18 with every other year displayed. Screenshot courtesy of dashboard created by OUSD Research, Data, and Assessment Department. (Click to dig deeper into the data!)

 

2. Historical Upward Trend: Increasing citywide cohort graduation rates over the past years show that all major racial groups are showing positive signs of overall improvement from 2009-18, mirroring statewide increases. (This citywide data is for all district and charter schools combined.) This also mirrors a nationwide trends in the graduation rate, which is currently at an all-time high of 83% of students graduating within four years (using the new methodology, see footnote). It’s worth noting how Oakland’s citywide average cohort graduation rate (75% in 2018) still trails the state’s (83% in 2018) and nation’s.

Figure 2. Oakland citywide cohort graduation rates by race from 2009-2018 **(See Footnote)

Caveats to my optimism:

  • 5-year and 6-year cohort rates: Currently, the gold standard is for students to graduate from a high school in four years. The downside of tracking only 4-year cohort rates is that it doesn’t capture the efforts and successes of students who need more time to finish their diploma for a variety of reasons, ranging from personal health issues to recent newcomer status to entering high school severely under-prepared. If a school is able to help those students graduate beyond the original 4-year window, they should be recognized for their efforts through 5-year and 6-year cohort graduation rates.
  • Who’s leaving the cohort? If students don’t graduate on time, they’re either a) still enrolled, which means they’re more on track to graduating, or b) have left school with little chance of graduating (also known as dropout or push-out rates). Our current accountability system is not built to capture the latter by focusing solely on graduation rates. A school with 70% of students graduating and 30% still enrolled differs sharply from a school with 70% of students graduating and 30% dropped/pushed out—but on the CA Dashboard, they would score the same on the 5×5 grid.
  • Not all schools are seeing same levels of progress. I bring up specifically Fremont because that’s the high school I would have gone to. That’s the neighborhood high school my little nieces will attend in 10 years. And if nothing changes and we objectively take the stats, they have just a 50-60% chance of graduating high school on time.
  • The racial gap in our graduation rates has persisted. As we saw in the graph above, African-American and Latinx students still lag behind their White and Asian peers in absolute graduation rates and are not progressing at an increased pace, i.e. closing the racial gap. If we are to close the gap, we must see a strong, accelerated increase in rates of African-American and Latinx students graduating.

 

As we turn our attention to graduation season, it’s important to celebrate our graduates’ achievements (Class of 2019, I see you!) and look at the numbers deeply to understand how we can get closer to a 100% cohort graduation rate. But equally as important, especially in our technologically fast-paced future, we need to make sure our graduates are prepared to succeed in college and career. More on that in the next post!

** So, prior to 2017, the methodology that California Dept. of Ed (CDE) used to calculate the four-year adjusted cohort graduation rate (ACGR) was not consistent with US Dept. of Ed requirements for ESSA reporting. Rather than going back and recalculating the ACGR for 2010-16, the CDE shifted to a new methodology for class of 2017 onwards. This means that the ACGRs for those two different time periods aren’t comparable and CDE strongly recommends against comparing rates from 2010-16 with those 2017 onwards.

What does the state’s lowest 5% list mean for Oakland?

On the last day of January 2019, with little fanfare and drowned out by the impending strikes over unacceptably low teacher pay, the California Department of Education published a list of schools that are eligible for additional support and funds from the state. It’s also the list of the lowest 5% of public schools in California as determined by the CA School Dashboard.

 

Thinking about quality
Oakland is over-represented on this list. Not surprisingly, I’ve received quite a few questions about the 5% list, especially as parents just last week had to confirm their enrollment choices for next year. The need for more quality schools is on everyone’s mind, so I’m here today to try to demystify and share some context.

Why publish this list at all?
Every Student Succeeds Act (ESSA), the latest reauthorization of federal K-12 education policy signed into law by former President Obama, requires that each state create clear criteria for identifying the lowest performing 5% of schools so that resources and efforts can best be deployed for improving student outcomes. After receiving feedback from US Department of Education during the ESSA state plan approval process, California chose to align the identification criteria around the CA School Dashboard, its current multi-measure accountability system for K-12 schools, inclusive of alternative schools starting Fall 2018.

CA School Dashboard Resources
Before we go too deep, let me share some resources on the CA School Dashboard. Check out the links for more useful information:

 

What are the criteria?
In California, the lowest 5% of schools are eligible for comprehensive support and improvement (CSI). If a school meets any of the following criteria on the CA School Dashboard, it will be identified as a CSI school: ·

  • Grad rates <67% over two years average (HS only) (example)
  • All red indicators on CA School Dashboard (example)
  • All red and orange indicators on CA School Dashboard (example)
  • All red except for one indicator of any other color (example)
  • Majority red indicators if they have 5+ indicators (think 6-12 schools) (example)

Identification for CSI is based on “All Students” group for both traditional and alternative schools and occurs once every three years. When the CA School Dashboard is updated each fall, a school can “exit” the list if they no longer meet the specific criteria they were flagged for. For example, if School X (grades 6-12) was flagged for low graduation rates but has a high graduation rate average over the next two years, they would no longer be flagged. They can be re-flagged in the future for other reasons, like having majority red indicators or if their graduation rates decrease again.

 

How’s Oakland looking?
So…which of Oakland’s schools are on the list? Here is a table ordered alphabetically:

Source: EdSource Database: California’s lowest-performing schools in 2018-19

 

Some quick analysis:

  • While the criteria are meant to yield a target of 5% of schools in California, Oakland is overrepresented on the list with 21% of its schools listed. (28 out of the possible 131 public schools. This includes both district-run + charter and alternative + traditional)
    • 24% of Oakland district-run schools were listed (21 schools)
    • 17% of Oakland charter schools were listed (7 schools)
  • Of the 781 total schools identified across the state, 4% of the schools are in Oakland.
  • 25% of the Oakland schools listed are non-traditional schools (e.g., continuation or alternative), which is similar to the percent of non-traditional schools listed statewide (29%).
  • About 2/3 of the listed Oakland schools were flagged for low performance and 1/3 were flagged for low graduation rates – also similar percentages to the statewide list.

 

Some grains of salt
Similar to the state’s roll-out of the chronic absence data for the first time in 2016-17, there’s going to be hiccups and imperfections whenever data is rolled out initially for a new metric.

  • Automatic Assignment of Orange. Schools that do not report their data are automatically assigned Orange in chronic absenteeism and suspension rate indicators. For example, we wouldn’t know whether School X truly ranks an Orange on suspension, or if they are Orange because the CDE did not receive their suspension data.
  • Data Errors. One local example of data error is Lighthouse High School. In 2017 they had a 75% cohort graduation, helping to flag their 0% graduation rate in 2018 as an error. Nevertheless, they’re on the list.

 

So what’s next? Time to focus on quality.
There are different levels of support based on how a school is doing (from highest level of support to lowest):

  • CSI = Comprehensive Support and Improvement (aka Lowest 5%. These are the schools who met the criteria above)
    • ‘CSI Low Perform’ = Flagged because they met one of the non-grad low performance criteria (e.g., all red on indicators)
    • ‘CSI Grad’ = Flagged because of low grad rates
  • ATSI = Additional Targeted Support and Improvement (when 1+ subgroup doesn’t improve and they’re a TSI school. I’m still not 100% clear on how this differs from TSI – if any readers know, please share with me at cchan@educate78.org!)
  • TSI = Targeted Support and Improvement (when 1+ subgroup doesn’t improve for 2+ years on same criteria), also referred to as General Assistance

Schools who qualify for CSI can submit improvement plans to California Department of Education to access state funds to support their improvement process. First disbursements recently went out to those with approved plans. As additional layer of support, county offices of education received extra funds to help support impacted LEAs with CSI schools (another way the Alameda County Office of Education is increasing its support to Oakland).

Locally, there are a lot of conversations, in offices and around dinner tables, about school closures and consolidations. (Data nerd here: I’ve been crunching historical closure data, and it’s a messy + sensitive topic. Curious if y’all are interested in seeing it?)

Instead, I hope this list along with other data can help drive more conversations around school quality. I’m more optimistic than ever we can get there, because this is the vision I consistently hear from OUSD Superintendent Kyla Johnson-Trammell. Also, with federal backing through ESSA, the state defining criteria for the list, and the county looking at differentiated support for these schools, we should all be able to align around a clear and fair definition of quality and invest in our schools to achieve it.

 

Additional links and data resources:

Are Oakland students making progress meeting state standards?

The budget is on everyone’s mind. So are the teacher contract negotiations. (Or maybe just winter break…) At times lost in the mix of all these serious issues is a central question for all Oakland public schools: Are our students making progress meeting state standards in math and language, the building blocks of learning? Well, here’s a table showing Oakland’s progress on the recently released state SBAC tests from earlier this year on math and ELA (English Language Arts):

Chart comparing Oakland proficiency rates on SBAC ELA and Math compared to statewide proficiency rates.

The answer: Technically, yes. Nevertheless, we’re nowhere we want to be – and we know we can do even better.

Oakland citywide on average is keeping pace with the state’s level of improvement in both ELA and Math. At this rate, however, it’ll take until 2040 for over 75% of students to be proficient in math and ELA.

 

The Painful Distribution of Proficiency

The slow progress is just an average across the whole city. Average schoolwide proficiency has a painful geographic distribution as well.

Last week, I attended an OUSD school board meeting where parents from The Oakland REACH turned out in full force to demand an “Opportunity Ticket” to get better access to quality schools. I can see where they’re coming from. I plotted schools’ SBAC proficiency on a map. And where I lived in Oakland when I was a kid, it’s surrounded by red dots – public schools where fewer than 35% of students are proficient.

 

SBAC proficiency rates of schools mapped across Oakland. Color corresponds from dark red to dark green, reflecting range from 0-100% proficiency rate on SBAC math or ELA test, respectively. Click on the photo to view the dashboard.

 

 

 

 

 

 

 

 

 

I’m glad that our city is improving on the whole (slow progress, but nevertheless upward progress. Wooooo!), and I also worry the most about the students in the city who need support and resources most urgently. Oakland Achieves recently conducted an analysis of African American students in 3rd–8th grade and how they did in on the CA School Dashboard (figures below). It’s truly shocking how few African American students are in schools in which students are performing above standard and/or improving. Particularly striking is how African American students are overrepresented in the category “Schools to Review,” which are schools that are scoring low on the SBAC tests and not closing the proficiency gap from year to year (aka falling more behind).

 

Screenshots from Oakland Achieves report on CA School Dashboard. Comparison of distribution of all students in Oakland vs. focus group on African American students.

 

What’s next:

So, I wrote this whole post on SBAC results…and SBAC proficiency is only ONE indicator of a school and student’s progress (albeit a very ­­important indicator). Our schools and students are so much more than just a score on a test (though positive progress is always a good sign and performance gaps amongst historically underserved subgroups are never a good sign). We and the state recognize this, hence the multimeasure approach of the CA School Dashboard, which combines a couple of key academic and culture/climate indicator. Good news: CA Dashboard data is out! And the Alameda County Office of Education, which is playing an ever more important role in Oakland public education, has a page dedicated to it with good resources, along with some additional dashboards from OUSD.

Keep an eye out for a post in 2019 as we dig into the results.

Happy end of 2018 everyone! I’m excited to dig into the data with y’all in the year ahead!

The Importance of Attendance…For Everyone!

As Superintendent Kyla Johnson-Trammell included in her back-to-school welcome message, attendance is super important. Inspired by her call for building positive attendance habits early in our young learners and since September was Attendance Awareness Month, I decided to dive deeper into the topic, specifically chronic absence, to help give context on why it matters so much – for everyone!

What is Chronic Absence?

Chronic absence is another way of looking at attendance by focusing on proportion of kids who missed a significant amount of school days, those who are chronically absent. For the first time, the state has defined how it will collect and calculate chronic absenteeism (see footnote for definition). This data is one of several indicators in the California School Dashboard that attempts to reflect school culture and climate and build toward a more holistic view of school performance.

In 2016-17 (the first year data was collected), Oakland’s citywide overall chronic absenteeism rate was 15.4% of students – that’s over 1 in 7 children across district and charter schools who miss 10% or more of their school days. For comparison, the statewide average is 10.8% of students in 2016-17 and recent analysis by AttendanceWorks puts the nationwide rate at 15.5% during 2015-16 (though they define chronic absence slightly different).

Why it matters:

  • For System Leaders: Because a lot of school funding in California is based on “average daily attendance” (ADA), not enrollment, a school loses approximately $85 per day for every student absent. This can add up quickly! For a district or charter with scarce resources and tight budgets (aka almost every school in Oakland), this feels important. To get an estimate of the potential minimum dollars lost, I did a quick back-of-the-envelope calculation. (Assuming 10% of school year = 18 days, 15.4% of Oakland’s 53089 students were chronically absent during attendance tracking period, and $15,337 per pupil funding annually in CA in 2016-17). That adds up to a potential loss of at least $12.6 million citywide! (Some students were absent a lot more than 18 days a year.)

18 days x(15.4% students chronically absent x 53089 students) x $15337 per pupil/180 days per year = $12.6M

  • For Students: Students who are behind and chronically absent are missing valuable instruction time – time that is necessary for closing the proficiency gap. When there are huge differences between subgroups, it becomes equitable access issue. To estimate the impact for socioeconomically disadvantaged students (SED), I made assumptions that: the 41.6K SED students citywide receive 6.5 instructional hours per day, attend school 180 days a year, chronically absent students miss at least 18 days (10%) of year, and used the citywide rate of 17.4%. That’s 849K total instructional hours lost in 2016-17. Every 1% citywide reduction in % of SED students who are chronically absent yields 49K extra instructional hours to work on closing the gap! Imagine all the reading or math that can happen in that time!

Chronic absenteeism rates citywide in Oakland by subgroup (CDE 2016-17)

  • For School Leaders: Attendance and chronic absenteeism rates can be leading indicators for a healthy school culture. Chronic absence is an early warning sign of academic distress, including school dropout (Oakland Achieves 2016). Unusual rates are symptomatic of other underlying issues and thus should be monitored carefully, acting as bellwethers for a healthy school culture. Some schools like Frick, MLA, and Hoover have been steadily reducing their chronic absence rates, showing consistent improvement not only in the past three years but also historically. They’re also schools with targeted and sustained climate and culture efforts from the school leaders.

Areas of Caution:

  • Focus on chronic absence rates beyond just the numbers. There are so many the factors that contribute to students’ chronic absenteeism, a variety inside and out of a family’s control. Health, transportation, family trauma, reactions to the national political climate – all this and more in Oakland. It’s necessary to understand the “whys” behind high rates and know each school site is different.
  • Metric not designed to capture change for individual students: Even if a school helps a student reduce the number of missed school days from 60 days annually down to 20, the student would still be considered chronically absent in both years because they’ve crossed the 10% threshold.
  • Differing definitions: The California Department of Education definition for chronic absence differs slightly than the federal definition (10% of school days vs. 15 days per school year). Chronic absence is calculated differently at continuation and alternative schools.

Snapshot of OUSD on the interactive map from The Hamilton Project, which displays federally collected chronic absence rates from 2015-16 on national, state, district, and school levels using the federal definition of 15+ days for chronic absence.

  • Reporting errors in first year: Not surprisingly, the first year of data collection for chronic absenteeism rates had its hiccups and inaccuracies as folks acclimated. Sites with flagged inaccuracies will have a banner to note this. See OUSD’s page for chronic absenteeism rates for example. (As a point of comparison: if you look at San Francisco’s, it’s much more wildly inaccurate.) Hopefully, data for 2017-18 will be more robust and accurate now that schools and districts have had time to adjust with first round of collection under their belts.

Looking forward:

I’m super excited (not surprisingly) about the possibility of updated 2017-18 chronic absence data in CA School Dashboard refresh (update scheduled late winter/early spring). In theory, chronic absence rates could be leading indicators, a bellwether to making sure students are getting the instructional hours they need, and we would have insight into how Oakland schools are progressing compared to statewide benchmark.

For those interested in learning more about chronic absence and ways to address it, I recommend checking out AttendanceWorks, a nonprofit dedicated to reducing chronic absence to advance student success, or OUSD’s Attendance webpage. Data geeks will appreciate the interactive map recently released by AttendanceWorks and The Hamilton Project at Brookings Institution on 2015-16 federal chronic absence data, allowing users to dig deeper into subgroup trends.

Footnotes: Definition: The California Ed Code defines a student as a chronic absentee if ”a pupil who is absent on 10 percent or more of the school days in the school year when the total number of days a pupil is absent is divided by the total number of days the pupil is enrolled and school was actually taught in the regular day schools of the district, exclusive of Saturdays and Sundays.”