A major assessment vendor selected ASG to help it update its financial systems and reports as it implements new company strategies. ASG will perform several important services for the company including analyzing its bidding process, implementing systems and procedures to track labor costs, defining requirements for a new time-tracking system, and developing financial reporting systems and capabilities. ASG has helped several assessment vendors better understand their costs, establish metrics, and implement financial reporting systems to better manage their businesses.
The process for conducting ASG’s 9th annual state assessment survey officially started today with ASG sending the participation letter to 50 state departments of education and the District of Columbia. Last year, 46 states and DC participated in the assessment survey which is designed to increase communication regarding issues of interest among the states. Participating states receive a database with all of the survey responses and can contact their peers in other states for further discussions. The complete database and a summary report will be issued by ASG in early 2020.
ASG completed its review of a major assessment vendor’s proposal writing process and delivered its report of findings today. ASG reviewed both winning and losing vendor proposals, as well as proposals from competitors in arriving at its findings. ASG developed a rubric to score proposals and provided both qualitative and quantitative feedback to the customer. The customer received a list of areas of strength, areas for improvement and red flag areas related to its proposals, as well as ways to improve the quality of its bids.
ASG completed its work assisting an assessment vendor in creating a metrics framework for its item development function. Item development is a major component of assessment system costs. The assessment vendor wanted to develop metrics for the functional area and selected ASG to assist it with the effort. ASG worked with the company to develop metrics frameworks for item development, forms construction and forms development.
ASG assisted the Tennessee Department of Education by reviewing the technical RFP submissions for its new Tennessee assessment system. ASG had previously assisted the department in creating the RFP for assessment services so was highly aware of the RFP content and the perfect choice to review vendor bids. The vendor bids were evaluated based on a defined rubric outlined in the RFP.
The Mississippi Department of Education selected ASG and its partner, WebbAlign, to conduct independent alignment studies for both its general and alternate assessments. ASG has worked with the department several times to develop its RFPs for assessment services and is intimately familiar with the state’s assessment system. WebbAlign is the premier organization conducting alignment studies according to the framework its founder, Norman Webb, developed years ago. The WebbAlign team conducted the alignment studies using Mississippi educators and independent experts and ASG reviewed the results and issued the final reports for the Department. The reports will be used by MDE in its federal peer review submission.
ASG completed a review of the item development function for a major assessment consortium. The review consisted of a complete evaluation of the development process steps, as well as a review of item development costs vs. industry averages. ASG documented each step in the development process and compared the process to industry best practices. Process improvements were recommended and potential cost reduction activities delineated.
ASG delivered a cost evaluation of the vendor bids for its new KPREP Assessment System to KDE. ASG used its industry unique Assessment Cost Model to complete the analysis. ASG also assisted KDE in writing the RFP for vendor services and during this process developed the vendor cost sheets for submission with the bid. ASG input the information contained in the vendor cost sheets into its assessment cost model to derive the appropriate vendor costs to develop and administer the assessment. The appropriate assessment costs are then compared against the vendor bid costs, on a function by function and line item basis, to complete the analysis. The final report provided Kentucky with solid evidence it could use in negotiating the final price with the selected vendor.
ASG completed three item development projects for a major assessment vendor. The projects consisted of a) an evaluation of its item development procedures, b) a cost analysis of its item development function, and c) creation of a custom bid template the organization can use to calculate its item development costs for customer bids. ASG used its knowledge of best practices in item development to review the vendors item development processes and its Assessment Cost Model to evaluate the vendor’s item development costs against industry metrics. Then, ASG created an interactive template the customer can use to develop its item development pricing for customer bids. The template provides the vendor with multiple pricing methodologies for developing its cost estimates including current vendor costs, anticipated future vendor costs and industry standards costs, each on a line item basis.
The Tennessee Department of Education (TDOE) selected ASG to assist it in preparing an RFP for the administration, scoring and reporting of its new assessment system, provide its proprietary Cost Sheets for inclusion in the RFP, and provide technical evaluations of the vendor responses to the RFP. ASG staff has written, evaluated and reviewed hundreds of state RFPs for assessment services and looks forward to working with TDOE in developing the RFP and evaluating the vendor responses.
Update – In December ASG completed its work for TDOE and the RFP is scheduled to be released in January 2019.
ASG recently completed two different cost analysis services for KDE. The first was an estimate of the costs for its new KPREP assessment system. KDE is operating under a firm budget for assessment services and wanted to make sure that the design of its new KPREP assessment system would fall beneath its assessment services cost cap. ASG evaluated the appropriate costs for the KPREP system and after recommending some minor changes to the assessment design, determined that the state could afford the new assessment system. Once this work was completed, ASG developed the vendor cost sheets for inclusion in the state’s RFP for assessment services. The cost sheets provide ASG with the information it needs to evaluate the vendor cost bids in response to the RFP. ASG had previously assisted KDE in developing the RFP.
ASG recently completed two important cost analyses for an assessment consortium. The first analysis consisted of building a fixed-variable pricing model the consortium will use to determine the appropriate amount to pay its assessment vendor as states and students are added to the consortium. The model separates the assessment into fixed and variable price components so only the variable cost components are calculated in determining the appropriate price increase as states and students are added to the consortium. The second analysis provided the consortium with a pricing model to determine appropriate customer service costs. The model uses inputs such as average handle time, average speed of answer, occupancy rates, availability rate and staff shrinkage rate to calculate appropriate customer service costs. The number of customer service representatives are calculated using the above variables and call center standard Erlang equations.
The process for conducting ASG’s 8th annual state assessment survey officially started today with ASG sending the participation letter to 50 state departments of education and the District of Columbia. Last year, 45 states and DC participated in the assessment survey which is designed to increase communication regarding issues of interest among the states. Participating states receive a database with all of the survey responses and can contact their peers in other states for further discussions. The complete database and a summary report will be issued by ASG in early 2019.
ASG completed its work with KDE and delivered the final draft of an RFP for assessment services for its new KPREP assessment. ASG worked with KDE and other industry experts to help design the KPREP assessment system and write the RFP for assessment services. The RFP included ASG’s vendor costs sheets. ASG has worked with KDE on numerous occasions over the past decade to perform cost analysis and RFP writing services.
The Florida Department of Education partnered with Assessment Solutions Group (ASG) and others to conduct an in-depth analysis of the feasibility and impact of using the ACT and/or SAT in lieu of Florida's statewide assessments as provided in ESSA. Would such a system meet the federal requirements for technical quality, adequate accommodations, and alignment to standards? Would it provide comparable data and fit into the accountability system? Representatives from Florida and the ASG team discussed the results of the study and implications for states.
The Every Student Succeeds Act (ESSA) has given states more flexibility and authority for implementing new assessment and accountability systems. Many states have taken advantage of this opportunity by creating Assessment Task Forces (ATFs) and/or stakeholder committees to review their current systems and solicit input around how to implement improved assessment/accountability systems. Some states have taken a very deliberate approach to the assessment/accountability system redesign and engaged a well-thought-out process to solicit feedback from educators, administrators, and the public on how to best assess their students. These states have used ATFs to gather extensive feedback and review their options for assessment/accountability systems. In this session, two states provided details on how they have worked recently with ATFs and ASG to gather input from key stakeholders and rethink how their assessment systems might be improved.
The Kentucky Department of Education (KDE) announced today that it has retained the services of ASG to assist it in writing two RFPs for assessment services, provide cost sheets for the RFP, and perform cost analysis of the vendor RFP responses for its next summative assessment. Kentucky and ASG have a long history of working together starting in 2010. The state has hired ASG to write each of its assessment RFPs and provide cost analysis services of vendor RFP responses since 2010. ASG completed its work in early 2018 on an RFP for College Entrance Exams and more will complete work on a major RFP for a new assessment program, to be issued by KDE in fall 2018.
ASG completed and delivered its report to the Florida Department of Education (FDOE) today on the Feasibility and Use of ACT and SAT in Lieu of Statewide Assessments on schedule earlier today. This important study included analyses of comparability, accountability, accommodations, alignment, and whether such an assessment system would be likely to pass UDED peer review. ASG managed the process and authored the peer review section of the report. Subcontractors University of Wisconsin (WebbAlign), Kansas University (Center for Assessment Accountability Research and Design), and University of Minnesota (National Center on Education Outcomes) completed the alignment, comparability/accountability and accommodations studies for the project.
The final report can be found at the following link: Final Report
The WIDA Assessment Consortium and ASG announced an agreement today to have ASG assist WIDA with several services in support of its ACCESS 2.0 English language proficiency assessment. The multi-year agreement will see ASG provide consulting services in a number of technical assessment areas, Information Technology consulting, RFP review services, and several different assessment cost analyses in support of WIDA’s ACCESS 2.0 assessment.
PAST ASG ACTIVITIES
(BOISE) – State Superintendent of Public Instruction Sherri Ybarra is continuing a statewide dialogue about what Idahoans want in their assessments for public schools.
“As I shared with educators last spring, a thoughtful conversation about what Idahoans want and value in an assessment system is long overdue,” Superintendent Ybarra said. “We are building on our successful Idaho Challenge, where Idahoans participated in a careful review of what we want students to know and be able to do. That resulted in the Idaho Content Standards, and now it’s time to talk about testing.”
The State Department of Education is teaming with Assessment Solutions Group throughout the 2017-18 school year to engage stakeholders -- from teachers to parents to business leaders -- in thinking and talking about K-12 testing.
“This isn’t a conversation about the strengths or weakness of specific tests,” Superintendent Ybarra said. “We want to hear from stakeholders what they value in assessment programs. We will share that information to guide the development of our state program to support student learning.”
A task force of representatives from stakeholder groups is expected to meet about five times during the year to review research and learn about how various assessments support teaching and learning. In addition, SDE Director of Assessment and Accountability Karlynn Laraway said, the department and ASG will engage other Idahoans through focus groups and gathering feedback through an online survey.
A report summarizing the task force’s work is expected to be shared with State Board of Education in June 2018.
The process for conducting ASG’s 7th annual state assessment survey officially started today with ASG sending the participation letter to 50 state departments of education and the District of Columbia. Last year 43 states and DC participated in the assessment survey which is designed to increase communication regarding issues of interest among the states. Participating states receive a database with all of the survey responses and can contact their peers in other states for further discussions.
The Indiana Department of Education will partner with ASG to determine the appropriate price for its new assessment system and issue the RFP for assessment services. ASG will use its proprietary cost model to determine the appropriate price for Indiana’s new assessment system and perform a technical review of the RFP for assessment services. Additionally, ASG will lead the technical review meetings of the state’s general and alternate assessment review committees.
The North Dakota Department of Public Instruction selected ASG to assist it in drafting its RFP for state assessment services, develop its proprietary Cost Sheets for inclusion in the RFP and provide cost analysis services of vendor RFP responses. ASG has saved states over $100 million in helping them analyze and reduce assessment costs.
The process for conducting ASG’s 6th annual state assessment survey officially started today with ASG sending the participation letter to 50 state departments of education and the District of Columbia. Last year 42 states and DC participated in the assessment survey which is designed to increase communication regarding issues of interest among the states. Participating states receive a database with all of the survey responses and can contact their peers in other states for further discussions.
The Education Alliance announced today it has contracted with ASG to conduct a project to determine the qualities of state assessments that West Virginians value, develop a set of key criteria based on those (and additional technical) qualities, and evaluate various publicly available assessments against the key criteria. The purpose of the project is to provide decision makers with objective information and a set of criteria and an analytical framework that can be used to evaluate potential state assessments.
A two-phased approach will be used for the project — Phase I: the identification of the key assessment criteria valued by West Virginia stakeholders; and Phase II: the analysis of available assessments’ strengths and weaknesses based on those criteria. Phase I of the project will include three methods of information gathering: (1) an electronic survey, (2) a series of focus group discussions, and (3) convening a stakeholder group. After stakeholder information is summarized, the second phase of the project will focus on conducting independent research on features and qualities of various national assessments currently being marketed and used by states to satisfy federal requirements under the Elementary and Secondary Education Act (ESEA), evaluating how each of these assessments rates against the key criteria developed in Phase I of the project, and presenting additional information West Virginia decision makers should consider when evaluating the state’s assessment options.
ASG and the Maine Department of Education announced an agreement today for ASG to review and provide input on its soon to be released RFP for assessment services related to the new Maine Educational Assessment (MEA) Math and English Language Arts/Literacy Assessment. ASG will review the RFP and provide input to ensure that the state’s fiduciary interests are protected. Additionally, ASG will provide its cost sheets for inclusion in the RFP for assessment services and provide cost analysis of the vendor proposals to administer the assessment. ASG has saved states over $100 Million since 2010 in providing cost analysis services.
ASG announced today that it recently completed its work for the Mississippi Department of Education in providing assessment cost analysis and vendor negotiation services related to the Department’s recently issued RFP for a new summative assessment in ELA and Mathematics and end-of-course assessments in Algebra I and English II. By using ASG to provide vendor cost analysis and vendor negotiation services, Mississippi was able to save $12 when negotiating the final contract price with the selected assessment vendor. ASG had previously written the RFP for assessment design and implementation services so it was well versed in the details of the Mississippi program.
ASG announced today that it will be moderating a 90 minute session on “lessons learned in implementing high quality assessment systems” during the June CCSSO National Conference on Student Assessment in San Diego. Four states, Idaho, Kentucky, Pennsylvania, and Ohio will present lessons learned in implementing their new high quality assessment systems in the recently concluded 2014-2015 testing year. A wide variety of topics will be covered including implementing high quality assessment systems that respond to states needs, using new technology delivery systems to assess students, working one’s way through political issues in developing new assessment programs, as well as implementing new assessment systems as part of an assessment consortium and as a single state.
ASG issued two reports outlining the findings of its 2014-2015 state assessment system to participating states today. The purpose of the survey is to give states visibility as to how their peers are handling a host of issues related to implementing the next generation of assessments in the 2014-2015 administrative year. The reports outline the qualitative and quantitative features of the state assessment survey that ASG has been conducting since 2011 and provide a wealth of data to participating states. The qualitative report covers topics such as how states are implementing new technology delivery systems, status of consortium membership, security concerns, other technology concerns, and quality issues in state assessments. The quantitative report provides a breakdown of participating state assessment costs, by subject area, consortium membership and assessment type. Roughly 40 states participate in the annual survey.