A New Year's Update from the Director of IES

Mark Schneider, Director of IES | January 8, 2020

The following is based on my welcome to IES' principal investigators meeting, January 8, 2020, at the Washington Marriott Wardman Park Hotel.

I think it will give the entire IES "community" a sense of our progress and our challenges.

Mark Schneider
Director, IES

I have been Director of IES for about 20 months. I want to begin by reviewing what we have accomplished in those months and then laying out some of the challenges that will occupy IES for the coming year and beyond.

What we have accomplished:

We have made a concerted effort to update our brand and improve how we communicate with our stakeholders.
This may seem like a strange place to start but it is essential to achieving our mission as an applied science agency.

Communicating clearly using plain English and helping teachers, administrators, and policy makers more easily identify programs and interventions that have a higher probability of working for them is essential.

Identifying what works is hard enough—but getting that information into practice is far harder and beyond the skills set of most of us (indeed of not only us but just about everyone and every organization). It's the infamous last mile problem.
I will come back to this theme several times but let me start with a few highpoints in our efforts to do better on this front:

We hired a new Communications Director, Beth Greene, who reports directly to me and is part of IES' senior management team, thus integrating a comms perspective into all our discussions.
We have a new logo, we are planning a complete rebuild of our website, and we are developing a unified vision of IES. Again, worrying about IES' "visual identity"—a new term for me—may seem like a distraction from our work as a science agency, but all this new work is designed to help us communicate our mission better.
Examples:

We are getting rid of over 100 logos in favor of just 1.
We are focusing on products rather than centers, so that you don't need to know which center produced a specific report in order to more easily find it.

We are working to end the era of long unreadable reports that were suitable for gathering dust; some of the longer reports could double as doorstops.

Instead we have guidance that calls for a 15-page maximum for reports accompanied by a one pager that should alert readers to the most important aspects of the report.

Supporting documents, tables, appendices will all be available on the internet with hyperlinks in the reports as needed.

We are still struggling with a 4-pager infographic format, but I haven't given up hope that we will find something that works.

We are working hard on plain language as well, but almost all of us have PhDs where the most uniform outcome of graduate training is difficult to read jargon filled writing that only another PhD can understand.

As I have said before, here's the entire style guide for IES: Short sentences. Strong Verbs.
It's easier to flag reports that are too long—which we are doing by moving to 15-page briefs rather than 400-page doorstops—than it is to flag reports that are badly written. But we are committed to identifying and fixing bad writing.

Consistent with SEER principles, we have increased our emphasis on long-term outcomes including employment and wages but also trying to assess the extent to which the effects of interventions persist over time. Here are two concrete examples.

We are now giving extension grants to researchers who are at the end of their current grant but have promising evidence.

NCSER has funded four of these this year and NCER will be funding some as well.

We are designing a competition that will incentivize research using state longitudinal data systems (SLDS)—with a focus on understanding how programs and policies at the state or district level relate to student outcomes over time.

We are investing more resources in practice guides, which are some of the most downloaded documents on our website.
They are essential to our mission since they help to translate research for arguably our most important stakeholders --- educators.

We currently have 5 What Works Clearinghouse Practice Guides in the works. Two are updates or extensions of guides that are more than ten years old, including one focused on students struggling with math and another focused on promoting social and behavioral success in elementary school. We're also tackling reading in the middle grades. And we have two new postsecondary guides in the works, including a guide on CTE in community colleges and effective advising practices.

We are considering how to maximize the work of our regional education laboratories:

RELs are our "boots on the ground," working with states and districts to get evidence-based policy and practice into the classroom. This involves translating research into useful products, professional development experiences, and diffusion activities designed to expose educators and policymakers to the latest in "what works."
If you have been following the blogs put out by our NCEE Commissioner, Matt Soldner, you know that IES is deeply interested in having RELs take an even larger role in scaling evidence-based practices in the coming years. We have learned a lot, but much of that still hasn't been turned in to materials educators can—and want—to access.
Some RELs and non-REL partners have done this really well: MOOCs, professional learning communities, and resources that help turn "what works" into "what happens" in the classroom.
We'll be asking RELs—and all of you—to help us do even more.

Increased emphasis on postsecondary work

We have broadened the focus on postsecondary education in NCER – applicants are now invited to apply under the topic that makes the most sense given their expertise or interest – and we invited applications for a Postsecondary Teaching and Learning R&D Center.
For the first time, NCSER is encouraging work in postsecondary education

We will continue to invest in systematic replications to build evidence on reading and math interventions developed and tested with IES funds that have shown evidence of efficacy.
The next round of systematic replications will expand from just IES funded interventions to ones that have evidence of effectiveness regardless of original funding.

What we are still working on:

We are strengthening research through implementation of the SEER Principles. As you know, these principles are embodied in each of our RFAs, and we already make several of the principles requirements for us to review an application, such as addressing cost effectiveness and pre-registering studies. Expect that to continue.
We know that we will need to continue to provide guidance and provide support to the field to make sure these principles are can be met.

For example, we have funded a grant to provide technical assistance to researchers who need to conduct cost analyses and cost effectiveness analyses as part of their grants.
NCEE is shepherding the process along and has awarded a contract to work through elements of SEER that need conceptual and methodological heavy lifting.

Core components of interventions and external validity are high on our list.

Our goal for each SEER Principle is a white paper with IES's "take" on the issue and an "entry-level" TA tool to help the field think about the Principle in their own work. As we begin to develop those first-generation products, we will be looking to fund other work that can extend them, such as providing more technical assistance in this area.

Giving non-academic experience for IES fellows.

It's good for researchers in training to understand the challenges and the opportunities of working in and with the people and organizations that deliver education services.
We increased our emphasis on apprenticeships in schools, colleges, SEAs, LEAs as a requirement for our training programs.

Identifying what works

Like almost all research across so many areas, we are plagued by a far too common pattern: we fund studies that cost millions, take years, and all too often fail to produce positive outcomes.
Despite some notable successes in identifying things that work, we suffer from a bottleneck: Medical research has dozens upon dozens of trials going to find a treatment for say Alzheimer's. These trials are funded by private companies and government agencies. We usually have one trial, funded by IES.

We must increase the number of trials we support. We need to test more and we need to fail fast. We need to identify many more promising interventions than we do now. And we need to get far better are understanding the conditions and circumstances that increase the probability that some intervention will work for identifiable sets of learners.

This leads to our intention to invest more resources exploring and supporting digital platforms that are often used by tens of thousands of students and teachers. There is an opportunity for researchers to use digital platforms to test some ideas/interventions more quickly and in a wide range of conditions in order to inform what we know about interventions.
We know that these platforms can do A/B testing and some other kinds of tests—but how far can they go testing more complicated ideas?

We will be encouraging more work in this area to see how far these platforms can help us achieve our mission.

There remain staffing challenges within IES – we are facing shortages that constrain our ability to do everything we want to do. That said, IES staff do an amazing job, including the work they put into making this conference work:

Thanks to NCSER and NCER Staff, in particular Christina Chhin, Katie Taylor, and Corrinne Alfeld who led the planning for this meeting
Also, thanks to the Meeting Co-chairs Nikki Edgecombe, Carl Sumi, and Venessa Keesler
And thanks to my senior staff, including Matt Soldner, Liz Albro, Joan McLaughin, and Anne Riccuiti—who do most of the heavy lifting on IES' research efforts.

Finally:

We are often asked "What keeps you up at night?" Here's my answer: the abysmal record we as a nation have regarding reading. This was evident in the recent NAEP and PISA results. But it's not just the poor scores that depress me—it's that we haven't made enough progress using known science, including rigorous brain science, in teaching and improving reading.

Consider Emily Hanford's work that shows teachers aren't using evidence and decades after the reading wars started and after decades of strong research—maybe the strongest research we have—over one third of American students are below basic in reading.
This to me is among the greatest challenge we face – if after so much time and so much money and so many RCTs and so much rigorous research, why is it that Johnny still can't read?
This keeps me up at night—and it should keep you up too, since it brings into question the effectiveness of our whole research enterprise.

Most of you know that I have used my blog to communicate issues and invite comments and I have appreciated your responses. Your comments have sparked debate among staff and pushed our thinking and planning efforts. I assure you I read all of them. Keep them coming.