We are excited to offer seven workshops at the 2023 Innovations in Testing Conference. During workshops, attendees learn and interact through structured, engaging, and thought-provoking group exercises.
Testing can provide important pathways to opportunity and success. As such, it is critical that testing programs actively seek to ensure test takers are not denied access to exams. At the same time, security protocols may have the effect of limiting access. For example, requiring an individual’s registered name match the name on their government-issued identification can exclude individuals undergoing a transition. Similarly, while proctoring can help ensure a level playing field for all examinees, the reality is that for many individuals, access to proctored exams can be a significant challenge. While reasons for this may vary – such as lack of current technology, inability to access stable internet service, the cost of travel, the health risk of testing with others, or the need to find a quiet location to test – offering limited delivery methods increases the barriers to testing for some individuals. These are just some of the challenges testing programs face as they seek to effectively balance access and security. Join experienced testing practitioners as they further explore these issues and the solutions testing programs are using to balance access and security in an effort to promote diversity, equity, and inclusion.
Speakers: Ray Nicosia, ETS; Faisel Alam, Law School Admissions Council; Bill West, Cornerstone Strategies; Elizabeth Azari, Achieve! Executive Consulting LLC
This workshop will be a collaborative session compiling information on existing — and still needed — research on alternative item types (AITs). While AITs aren't inherently better, we do know that they can affect item performance characteristics. We will start the workshop with a brief review of what is already know about AITs and then open a group discussion, inviting all the participants to share their experiences with AITs, including where they see an important lack of evidence-based measurement. We will compile open questions and then break into small groups to discuss these ideas further. We will also compile success stories about general characteristics of content that work well with certain response formats. The goal will be to collect information from attendees and to solicit colleagues to participate in a post-conference paper, special topics journal, or book. At the end of the workshop, we will solicit co-authors and make an initial plan for continuing the work post-ATP.
Speakers: Kirk Becker, Pearson Vue; Ruth Ramstad, Pearson Vue; Belinda Brunner, Inteleos; Cynthia Parshall, Touchstone Consulting
This session will focus on providing a legal overview and practical considerations for “best practices” when reviewing requests for disability-related accommodations. The presenters—seasoned experts in disability law and disability accommodations for high-stakes testing—will provide many examples to help illustrate each issue. The presenters will present an approach that is balanced and reasonable, that takes into consideration both the reasonable access needs of candidates with disabilities, as well as the need to adhere to the law and be fair to all candidates. If your organization has struggled with thorny accommodations requests, or you are ready for a comprehensive review of the issues, this session is for you.
Speakers: John Hosterman, Paradigm Testing; Donald Balasa, American Association of Medical Assistants; Heather Case, International Council for Veterinary Assessment
Remote desktop and screen-sharing technology that can go undetected by test delivery providers’ system scans give bad actors unprecedented access to exam content and deliveries, threatening the validity and value of exam results as never before. Likewise, the human and financial resources that go into developing exams is at greater risk because of online testing. Using data forensics to identify anomalous testing results, investigating how they were obtained, and taking corrective action against the candidates who are identified, is an effective way of turning detection into prevention and deterrence. This workshop demonstrates the process of designing and developing an effective data forensics framework for a certification program, which allows the testing organization to screen data daily and justify corrective action. We will cover a basic analysis you can do of your program, the major framework components, and share experiences on developing practical statistical indices and IP geolocation investigations and taking corrective action when a result is flagged. We will also introduce next generation ML-based data forensics, including research design and preliminary findings.
The goal of this workshop is to help practitioners gain insights about establishing a data forensics program that will work for their organization and how they can use data forensics in conjunction with other investigative techniques to protect the validity and value of their exams.
Speakers: Huijuan Meng, AWS; Jason Bell, AWS; Ye Ma, AWS; Kirk Becker, Pearson VUE
Test item writing is critical to ensure that items are culturally responsible and inclusive. Training subject matter experts to recognize, understand, and appropriately introduce characteristics of race, ethnicity, culture, and identity in test items can provide evidence of exam fairness. This workshop will guide participants through peer review training aimed at uncovering unconscious bias. First, facilitators will provide participants with background information. Next, participants will split into groups. Each group will participate in peer review training and will apply best practices for developing culturally mindful test items. Then, participants will practice the process of peer review in their groups to identify and correct bias in test items. Finally, everyone will come together to share their results, discuss the implications of different training methods, and learn how this process can be complemented by post-exam statistical analysis.
Speakers: Kim Cavanagh, Gannon University and PAEA; Nemanja Dukic, Alpine Testing Systems; Denise Rizzolo, PAEA; Brett Foley, Alpine Testing Systems
When incidents arise, whether related to exam security or personal data loss, it can seem unmanageable. Often, incidents can include multiple departments and your board internally and multiple vendors, test takers, media contacts, and even law makers externally. With so many interested stakeholders, managing incident responses can be difficult. In fact, addressing so many diverse opinions and agendas can at times seem unmanageable. During this workshop, we will discuss how you can position your organization for positive incident response results. Building together, we will explore incident response planning, communications preparedness, and investigation tips that will position your organization for positive results. Presenters will call on real life experiences and share keys wins and learnings that you can use to manage incidents for your organization.
Speakers: Camille Thompson, The College Board; James Tunkey, I-On Asia; Shari Lewison, Cornerstone Strategies
This workshop will begin with a discussion of Microsoft’s journey to creating a delightful exam experience that is only possible through a close partnership with ITS and Pearson VUE who worked together to identify and prioritize the challenges that are preventing the experience from being delightful and to develop solutions. We will share lessons learned along the way, including best practices and tips, that you can use as you embark on your own accessibility journey. With that overview in mind, we will then ask participants to share the accessibility challenges that their programs are facing or those that they believe they will have to resolve. After brainstorming these challenges, we’ll vote on the top 5 challenges to tackle together during the workshop. Depending on the size of the audience, we will assign each challenge to one or more groups and ask them to brainstorm possible solutions that programs can implement to address those challenges or additional research that could be done to better understand how to address the problem from a “delightful experience” perspective. Each group will summarize the results of their discussions with the goal of creating a high-level implementation plan that attendees can leverage in their own programs as they, too, move toward more accessible, delightful exam experiences that benefit not only people with disabilities but all candidates.
Speakers: Liberty Munson, Microsoft Corporation; Don Tanedo, Microsoft; Pat Hughes, Internet Testing Solutions; Joan Lambert, PearsonView the Full Program