Outsourcing SBAC Scoring

by Kathleen Jeskey

Last Sunday, Oregon Save Our Schools and BadAss Teachers (BATs) posted by the Oregon Department of Education (ODE) about the upcoming Smarter Balanced Assessment (known in Oregon as SBAC).

Following that presentation, another member of Oregon Save Our Schools, Portland parent Siobhan Burke, wrote an email to ODE. In her email, she cc’d other members of Oregon Save Our Schools who were not in attendance at the meeting, including Hyung Nam, a Social Studies teacher at Wilson High School. Hyung has been on the SBAC and the other new Common Core assessment, PARCC.

Following is the email exchange between our members and the Oregon Department of Education:

On Wednesday, February 25, 2015 8:56 PM, Siobhan Burke wrote:

“Hi --
At last night's presentation we were told that AIR was subcontracting work for for the open ended questions on the OR SBAC out and would be hiring certified teachers to score. Parents are very concerned about this and we would like documentation of this fact. Is there contract language or a job notice you can provide to back up this claim. Thanks in advance for your speedy reply. Sincerely,
Siobhan Burke PPS parent”

Hyung Nam responded:
“I can't believe how negligent policy makers and administrators are. I looked into it in just a few minutes and found this.

Kristin Gimbel at OEIB confirmed that the test vendor (AIR) is subcontracting with a company called Data Recognition Corporation, DRC, to manage scoring of constructed responses.  They operate in much the same way the Measurement Incorporated and Pearsons scores the tests using random temp workers paid low wages. DRC is paying $13/hour, like Pearsons while MI pays $10.70. They all make the only qualification, a BA or 4 year degree in any subject. These low wage temp workers are not educators and not qualified to evaluate our students. They work under conditions that are not conducive to good assessment, sitting for hours reading huge volumes writing of students they don’t know.

Furthermore, this company has ties to the conservative corporate lobby, the American Legislative Exchange Council, ALEC.

And finally, this type of scoring by temp workers for this very corporation is invalid. Here is a story about the sausage making that goes on.  “

Doug Kosty (ODE) wrote

Thank you all for expressing your concerns on this important matter.

The information you received from Kristen Gimbel partially addresses the question raised in Siobhan’s original email. However, I’d like to clarify that Derek did not state that DRC would be using certified teachers for scoring the Smarter Balanced assessments this year during the meeting.  That said, please accept ODE’s assurance that the approach used by DRC does not compromise the validity of the assessment scores.  DRC follows industry standards and best practices for scoring large-scale assessments.  Raters are required to possess a 4-year degree in the content area they are scoring, all applications are screened for relevant educational background and work experience, and are interviewed and assessed as part of the application process.  All raters will be trained and certified prior to scoring actual student work.  Lastly, raters will be monitored for accuracy through the scoring process.

Let me also assure you that my staff will be monitoring the first year results carefully to ensure that the scores are valid before they are published for schools to view. 

ODE agrees with you that, ideally, scoring of the Smarter Balanced assessments will be done by Oregon educators similar to how we have scored the state Writing assessment in past years. Engaging Oregon educators in the scoring process provides rich professional development for teachers and serves as an opportunity for teachers to deepen their understanding of the content standards. Given all the logistics involved in getting this first operational year of the Smarter Balanced assessments up and running, however, it was not feasible to adopt this scoring model for these first years.  Moving forward as indicated at Tuesday night’s event, ODE will continue to pursue options to include Oregon educators in scoring the Smarter Balanced assessments in future years. 

Thanks again for your email and for everything you do for our children.

dk

Doug Kosty

Assistant Superintendent | Office of Learning |Instruction, Standards, Assessment, & Accountability|Oregon Department of Education  

255 Capitol Street NE | Salem, Oregon 97310
É 503.947.5825 | 7 503.378.5156 | *   |

The phrase that bothers me is “DRC follows industry standards and best practices for scoring large-scale assessments.” Is it okay with you to have students right out of college score your child’s work as a temporary, $12 an hour worker as opposed to their classroom teacher, who is trained and experienced and can ask your child clarifying questions to truly get at what he or she knows? Is it okay with you to have high stakes decisions about your child her school, or her teachers made based on those scores?

Is it okay with you that at the Gladstone meeting when asked specifically if “ONLY EDUCATORS” would be scoring our students’ work,  ODE’s Derek Brown answered “Yes”,  when clearly that is not going to be the case?  Is it okay with you that Oregon has not yet had an honest conversation about these assessments?

The next ODE presentations on Smarter Balanced are as follows. Please attend and ask questions. 

Hillsboro: March 9, 6:30-8:30 Hillsboro High School

Albany: March 11, 6:00-7:30 Albany Public Library

Neah-Kah-Ne: March 12, 6:00-8:00 Neah-Kah-Nie High School