CourseGateway: They knew the solution all alongAuthor Phil Hill /6 Comments/by Phil Hill
I have a feeling that a blog series is developing.
Two weeks ago I wrote a post about “The Gates Foundation Messaging Machine,” where despite claims of following data, the reality of the research release was all about the messaging desired by the foundation. I asked:
Wait – is this a research report trying to understand perspectives of those who chose to not to pursue a degree, or is this an advocacy post with pre-ordained outcomes?
Late yesterday EdSurge published an article about CourseGateway, titled “New Effort Hopes to Make ‘Weed-Out’ Courses More Equitable,” clearly leveraging the interest in the NYU Organic Chemistry debacle as well as the general concern over disadvantaged groups’ performance in these critical gateway courses. And this story is also about the Gates Foundation’s desired messaging. The setup:
Research by the nonprofit Gardner Institute and other groups has found that these typically-freshman-year classes tend to prune some kinds of students from academic tracks more than others. For example, a new study published in the journal PNAS Nexus found that, among students who perform poorly in intro STEM classes, those who are underrepresented minorities are even less likely to end up earning STEM degrees than their white, male counterparts.
And the pandemic may be exacerbating this situation. Research out this year from consulting firm Tyton Partners, the Bill & Melinda Gates Foundation and Every Learner Everywhere found that professors lately are reporting an increase in the DFWI rate—that is, the percent of students in a course who receive a D grade, fail, withdraw or don’t complete the class.
The problem is the increase in DFWI rates of gateway courses, with a particular emphasis on underrepresented minorities. That is the issue at hand – a bad situation that needs to be addressed.
But the article then adds this whiplash-inducing transition from the problem to the proposed solution.
“Statistics show significant inequities in the completion rate in gateway courses,” says Andrea Jones-Davis of Educause. “Research shows students may drop out or change their major based on these gateway courses.”
Jones-Davis is the director of a new effort, CourseGateway, which aims to help more students—and especially Black, Latino, indigenous and low-income students—succeed in the first few classes they encounter in higher ed by promoting the adoption of high-quality courseware, a category of tech tools that packages digital reading materials with assessments and study supports. The theory is that courseware has the potential to improve student outcomes because it tends to offer students personalized instruction and instant feedback on their work, Jones-Davis says. These tools can also help instructors track whether and how students are studying and completing assignments.
Apparently, there was insufficient curiosity to ask the obvious question: what is the basis for your belief that promoting the adoption of high-quality courseware will actually help more students – and especially Black, Latino, indigenous and low-income students? The rest of the article goes into descriptions of how this courseware promotion intitiative will function. The key point is that the CourseGateway initiative is based on a product review site that aims to “provide enough information so that higher ed educators can select products for use in courses.”
Promotion is the key word here, both in who is doing the promoting and what is being promoted. Educause received a 30-month grant last year from the Gates Foundation for $4,945,954, with CourseGateway being one of the key deliverables. Who is promoting? The Gates Foundation through Educause with an assist from EdSurge. For those attending the Educause conference in two weeks, I am quite confident that you will hear more about this initiative in Denver. ISTE, the parent of EdSurge, is working on two separate grants from the same foundation for $3,180,033. Those grants are focused on K-12 and not higher education, but they should have been disclosed with the article.
As to what is being promoted, the answer is clear and historically consistent, “the adoption of high-quality courseware.”
Searching for a Connection
Even if the article won’t address the question about the connection of the problem and the solution, we can.
The Gardner ($600,000) and PNAS Nexus cited research directly touch on the question of underrepresented minority performance in gateway courses, as stated, but that is not the issue at hand. The Tyton Partners research ($10,381,410 under three grants, with the two postsecondary ones ending around the time of the stated research release), however, does touch on the solution of courseware.
Tyton, along with Bay View Partners and everylearner everywhere (the Gates Foundation funded initiative under WICHE, which has $2,909,910 funding under three grants) wrote the report “Time for Class: The state of digital learning and courseware adoption,” and it is based on a Spring 2022 survey of “approximately 850 administrators and 3,200 faculty at 1,200 unique postsecondary institutions and interviews with over 15 instructional materials and digital learning providers,” focusing mostly on introductory gateway course usage. Like most of Tyton-led research, it is very well-written, organized, and documented. I’ll admit some jealously in how well they present their reports. So far, so good – let’s hear from educators in an organized fashion.
The quote used in the EdSurge article is accurate (increased in DFWI rates), but that does not show any connection between problem and solution.
What is apparent from reading the report is that it is research in the sense of product management research around courseware. Let’s analyze adoption, look at use cases, look at Net Promoter Scores, align to known evidence-based teaching practices, and finish up with recommendations to improve the product category. And it is honest product management research that does not artificially inflate the value of courseware. In fact, the organized view of educators does not support the claimed connection between high-quality courseware being a solution to inequitable gateway course performance. In current use cases for courseware, the bottom two listings are improving outcomes for students with financial need and improving outcomes for Black, Latinx, and Indigenous students.
A good section in the report is based on comparing non-courseware adopters to courseware adopters in their usage of evidence-based teaching (EBT) practices. In this view – courseware adoption seems to be negatively associated with these practices, with only three out of 10 practices increasing for courseware adopting educators.
The report then compared courseware to e-texts to a new product category – advanced e-texts. The idea of this new set of products is to add a few pieces of functionality to a static e-text (auto-grading, enhanced LMS integration, practice questions, etc). When looking at adoption of evidenced-based teaching practices, it is the new advanced e-text product category that shines, not courseware, most likely due to product simplicity.
I’ve Got a Fever, and the Only Cure is More Courseware
The CourseGateway initiative, like most of its Gates Foundation funded predecessors, makes nebulous claims without any real basis. On their about page:
When implemented well, digital learning demonstrates the power to close equity gaps in gateway courses and throughout the curriculum. Two crucial components of this process are building awareness and adopting high-quality, equitably designed courseware that improves student outcomes.
Ouch, that’s some slippery logic. Digital learning can solve it all, but that includes many product categories. But they declare that building awareness and adopting courseware are crucial and produce results.
Really? Back up that claim if you want us to believe that this initiative will produce any results. The Tyton report funded by the Gates Foundation suggests that product review of courseware might not be the answer, at least based on the input from real educators teaching real students. And there is no other research or, you know, evidence presented that suggests that courseware adoption as an intervention will improve the DFWI trends of underrepresented students in gateway courses, aka “the problem to be solved.”
Disclosures: Lumen Learning, which has courseware being evaluated through CourseGateway, has been a long-time client of MindWires. MindWires used to be a grantee of the Gates Foundation from 2013 – 2018.
“ MindWires used to be a grantee of the Gates Foundation from 2013 – 2018.”
I think this fact, and why the relationship began, ended, and what purpose it served, would be a great candidate for the next article in your Gates Foundation series.
Jeff – I think you’re right. I just need to convince myself it is not navel-gazing or talking out of school.
Phil, your articles consistently provide thoughtful analysis backed by meaningful data. This advocate of technology in education is grateful to have access to fair commentary like yours.
Hi Phil – I think you know that I am the Senior Program Officer working on these courseware-related initiatives. I welcome the opportunity to speak with you about our goals and answer any questions you have. At the core, we are committed to helping Black, Latino, and Indigenous students, and students experiencing poverty equitably achieve a quality credential or degree. One (of many) focus areas is working with institutions, organizations, associations, and people who are developing innovations in equity-centered teaching practices and digital learning tools and curricula. I would encourage you to dig into the good work of Every Learner Everywhere. With specific regard to Coursegateway, it is simply a tool to help faculty discover, evaluate, select, and implement quality digital learning tools and implementation best practices. This is an extension of the work we did together on the Next Gen Courseware Challenge and courseware-in-context framework. The Tyton research you reference above is the longitudinal research we support and make available to the field to see trends in teaching and learning. The EBT research is one way to look at the impact of digital learning curriculum tools like courseware and how it supports faculty practice. The use of EBTs is not a measure of the ultimate efficacy of courseware in closing gaps. We also continue to invest in educational research to help build the evidence base for what works in what context with our focus students. Hope this helps clarify. Feel free to give me a shout if you want to discuss this further. We deeply value your insights and expertise…
Hi Alison, good to hear from you and thanks for the comment. I appreciate the offer to answer any questions I have, as well as the description of goals in the comment. I think it would benefit the community to do this in public, however. Would you be open to an email Q&A that would then be collected into a blog post? You could also provide an introduction (perhaps your comment above) that would be included with the Q&A. Let me know.