Virtual Spring Summit 2022

April 26 & 27, 2022

Summit Schedule

The Performance Testing Council is supported in part by our Cornerstone Members:

Cornerstone 21-22.jpg

This Summit is sponsored by:

SP22 Sponsors.png

Certification in the Metaverse

Wallace Judd, PhD

In March, 2022 ANSI (American National Standards Institute) awarded the very first accreditation to a VR certification – the Construction Hazards exam by ITI (Industrial Training International). In the beginning, Construction Hazards IDentification (CHID) was just a VR 1st person shooter game. This presentation will highlight several of the issues that arose when we tried to turn this ‘game’ into a psychometrically sound ‘exam’.

Key Objective:

• To highlight how to create a VR test that meets EEOC guidelines for employment, promotion or retention.

Wallace Judd, PhD

President, Authentic Testing Corp


Dr. Judd is a psychometrician who received his undergraduate degree from Princeton, his M.Ed. from Harvard, and his Ph.D. from Stanford University, doing his doctoral dissertation under Lee Cronbach on computerized adaptive testing.

He worked on the Lisa at Apple, was a senior programmer at Xerox PARC, and was Director of Training for Netscape.

As president of ComTrain, Inc., he developed the Judd Test series of ten computerized hands-on performance tests, starting in 1989, and has since developed or consulted on dozens of performance tests for clients, including Kelly Services, Oracle, Honeywell, and many others.

The former Executive Director of the Performance Testing Council, Dr. Judd is Chair of the subcommittee which developed a national standard for Performance Testing, ASTM E2849. He is also a former ANSI assessor for ISO/IEC 17024 and has done extensive research on performance testing standards, implementation and psychometrics. He has previously consulted with the American College of Surgeons on certification for Robotic Surgery. Author of 20 textbooks and numerous journal articles, he has two patents and is currently the President of Authentic Testing Corporation, a small performance-testing consultancy. Authentic Testing clients have included 18 major certification bodies, including among others Red Hat, Linux, Cloud Foundry, the Open Stack Foundation and Crossfit.

Spinning Up Virtual Machines

Anna Rubin

In an age of digitization that is causing waves of disruption across all business verticals and market segments, it is evident that the eld of IT skills validation and assessment, exams and test-taking methods has to evolve. It is all too easy to get caught up in old paradigms and we must contemplate new models that enable digitization of test delivery systems. This session will explore developing and delivering performance-based, lab experiences for exams by spinning up virtual hardware within the test driver.

Key Objectives:

• Understand lab based testing experiences

• Understand the development and delivery challenges

• View lab experiences within an exam

Anna Rubin

Manager of Exam Development, Cisco

Anna Rubin is the Manager of Exam Development for the Global Certifications organization of Cisco where she oversees the development, delivery and maintenance of over 100 of Cisco’s Associate, Professional and Specialist exams. Anna is a certification and workforce development expert with 30 years of experience helping organizations manage and grow their certification, accreditation and recognition programs. Prior to joining Cisco, she was Vice President of The Communicators, a certification consulting rm, where she led associations and organizations through the certification design and delivery process, market research and gap analysis, and development of education and corporate partnerships. Anna’s experience also includes directing ACT’s Washington, DC office where she oversaw government affairs and partnerships with education and workforce development organizations. During her time with ACT, she also managed scholarship and recognition programs for higher education. Anna began her career with the U.S. Department of Education working on education policy issues and directing the U.S. Presidential Scholars Program. Anna is a graduate of the University of North Carolina at Chapel Hill and is active in the Association of Test Publishers and the Institute for Credentialing Excellence.

Performance-based testing lessons learned during the pandemic - the Certiport story

Ty Kenworthy

Now in its third year, the COVID-19 pandemic has upended lives and businesses in countless ways—with academic operations experiencing some of the greatest disruptions. By extension, the performance-based testing (PBT) model was challenged on multiple fronts, giving rise to some serious and legitimate questions. What’s the future viability of PBT? How much value do customers and test-takers place in it? Will the risks illuminated by our pandemic response outweigh the risks of transitioning to traditional exam formats? For a business built on PBT, these questions triggered more than a bit of soul searching.


Without giving too much away, we’re still delivering performance-based tests today. Join this session to understand the reckoning one deeply invested PBT company made over the past two years. We’ll share some of the answers we’ve settled on, as well as our perspective on the future of PBT in an indelibly changed (and changing) world.

Key Objectives:

• Learn about the future viability of PBT
• Discover the value of performance-based exams to customers and test-takers
• Gain knowledge of test delivery challenges within the higher education market

Ty Kenworthy

Business Development Manager, Pearson VUE

Ty Kenworthy is a Business Development Manager for IT professional and academic clients with Pearson VUE and their Certiport business. Ty also serves in a secondary role as the manager of Research Services for Pearson VUE’s IT certification business and academic channels. In his 12+ years in the IT certification industry, Ty has held roles in exam development, psychometrics, business analysis, strategic planning, and market research. This varied background and experience has given him a wide view of the industry’s strategic and operational functions. Ty received his degree in Cultural Anthropology from Brigham Young University, with an emphasis in qualitative research methods.

15 years in. Developing and maintaining a computer-based simulation exam

Craig Simms & Kevin Leonard

The International Joint Commission on Allied Health Personnel in Ophthalmology (IJCAHPO) developed two simulated computer-based examinations to transition from traditional hands-on skills performance testing to a computer-based simulation testing environment.


The examination was developed for eye care professionals to enhance patient care by extending the capacity in the diagnostic testing environment, assist in working with more difficult or advanced patient issues, and the capability to rely more on evaluation than instrument feedback.


Subject Matter Experts consisted of eye care professionals who assisted in the development of the simulated examinations. The examinations included a multiple-choice and simulated component as part of an occupational certification/licensure that was determined to be job valid and reliable.


Technologies including Unity were employed to render the simulations within test drivers and learning management systems.

Key Objectives:

• Transitioning from hands-on testing to computer-based simulated examination

• Utilization of SMEs to develop reliable and valid simulated examinations

• Psychometric analysis and scoring protocols

• Selecting the right technology, updating simulations, integrating with your test driver and examination implementation

Craig Simms

Chief Learning Officer, IJCAHPO

Craig Simms is certified by IJCAHPO as a Certified Ophthalmic Medical Technologist (COMT), Certified Diagnostic Ophthalmic Sonographer (CDOS), and Registered Ophthalmic Ultrasound Biometrist (ROUB). He is the Chief Learning Officer for IJCAHPO and is responsible for developing IJCAHPO’s educational programs and services for Allied Ophthalmic Personnel (AOP). He lectures and conducts skill-based training throughout Canada, the United States, and internationally for hundreds of AOP annually. He has authored several journal publications on the AOP workforce shortage, training program curriculums, and technician training on ophthalmic technical equipment patient examinations. Simms is a founding member and past president of the Canadian Society of Ophthalmic Medical Personnel (CSOMP).

Kevin Leonard

Director of Business Development, Breakthrough Technologies

Kevin Leonard has worked in web technology since 1998. At Breakthrough Technologies, Kevin helps organizations bridge the gap between classroom learning and real-life experiences, allowing professionals to apply their knowledge in a dynamic and engaging simulation where users can manipulate equipment and interact with patients as they do in their day-to- day profession.