Program Performance Evaluation Plan
The State of Indiana Office of Early Childhood and Out of School Learning (OECOSL) received a large federal Preschool Development Grant (PDG) consisting of multiple projects intended to strengthen Indiana's early childcare and education system. The Early Childhood Center (ECC) worked with the OECOSL to develop the performance evaluation framework and process included in the original grant application. Once the Preschool Development Grant was awarded to Indiana, OECOSL contracted with the ECC to conduct the PDG's Program Performance Evaluation. ECC staff and PDG stakeholders have worked together diligently to ensure that the evaluation plan is responsive to and aligned with Indiana's vision. The PDG is composed of 22 individual projects geared towards developing a unified birth-5 system. Collectively, the projects promote community-led systems, incentivize high-quality programs and supports, build a highly qualified workforce, and create an infrastructure that supports information sharing that eliminates duplicate efforts. The size and scope of the PDG require the IU evaluation team to work closely with the state PDG project manager and individual project managers to ensure that projects make continued progress and that their outcomes are reflected accurately.
We worked with PDG project teams to develop logic models defining their expected outcomes and work plans outlining their timeline. Teamwork (link), an online project management tool, was adopted to track projects' progress, house pertinent documents, and inform stakeholders of changes. It is easily accessible, user friendly, and provides stakeholders, decision makers, and the Governor's office access to individual projects. Project managers were given individual Teamwork tutorials introducing them to its functions and are responsible for updating their monthly progress. Due to the size and breadth of the PDG, the ECC compiles monthly summaries of PDG projects' completion percentages and overall progress to ensure accurate data which is then used to inform the reports to the governor's office and the federal quarterly reports.
Eight of the 22 PDG projects are developing important models or practices for improving access to or quality of early childcare and education, such as Kindergarten Transition and Transition to Teaching. When considering models and practices for statewide adoption, they need to be examined through a lens that considers whether they are research-based and can be realistically implemented in Indiana after understanding the needed resources and supports. This information is crucial to state decision makers. The program performance evaluation process involves reviewing supporting documentation, conducting an evaluation interview, completing an evaluation rubric * checklist (link), and assigning a final recommendation. The evaluation findings will inform the State and assist it with making decisions concerning the adoption of models or practices.
The ECC will provide a final report to state decision makers that includes summative Teamwork data, details of the evaluation process, and final recommendations regarding the adoption of the PDG projects' models and practices.
*The checklist was developed from the Early Childhood Specialist Toolkit: Hexagon Discussion and Analysis Tool for Programs and TA Providers by the National Center on Early Childhood Development, Teaching and Learning, 2018 by NIRN (National Implementation Research Network).
