A new piece of equipment has found its way into the surgical facility of the Upland Animal Shelter. It’s not a gripper or a cutting tool, or even a laparoscope. It may be, however, the future of clinical skills assessment for the College of Veterinary Medicine.
The new piece of equipment is a generic tablet device resembling a classic Apple iPad. In early August, Dr. Ohad Levi, DVM and Maria Fahie, DVM, MS, DACVS piloted the first use of these tablet devices to electronically record students’ skill level while performing simulated canine sterilization, more commonly known as neutering or spaying. They were assisted by Registered Veterinary Technicians Lauren Kreisberg and Kathryn Gresco.
The inspiration for a tablet-based approach to assessing students’ surgical skills grew out of Vet Med’s need to make the testing process and handling of student performance data more efficient. Dr. Ohad Levi, DVM, described how the testing process has been traditionally conducted. “Our Small Animal Surgery/Anesthesia course is the first time our third year CVM students get hands-on experience as a primary surgeon. But before they operate on live animals, we have to make sure they have the necessary skills. In the past, we used a paper form to score each student’s ability to locate the anatomical necessary areas, find the best site for the incision, and properly suture the wound. There are about 30 skill areas we have to assess.” These paper forms were then collected and sent to the course leaders, who translated the handwritten scores to an electronic spreadsheet. “It used to take us 20-30 minutes per student to transfer the proficiency ratings and calculate final scores, so with over a 100 students, the amount of faculty time needed to complete the process was enormous.”
Working with WesternU’s Instructional Technology & Distributed Learning group, Drs. Levi and Fahie provided an Excel version of the form they used to manually record student scores. Web Technologists Esther Davis and Danny Vargas then translated that form into an interactive Web interface that could be displayed on the tablet device. Danny: “Our first challenge was to set up LDAP (Lightweight Directory Access Protocol) so that instructors could access the form using their standard WesternU username and password. Then we had to create a database with the records of CVM students who would need to be tested. Our initial interface was a straightforward list of assessment criteria and proficiency levels, but after some trial and error, we realized that a drop down menu for each assessment criteria was a more efficient way to record student proficiency scores.”
Once the form is completed, the changes are saved, the final score computed, and the data are sent to the appropriate column in the Blackboard Grade Center. If the Internet connection is lost, the data is stored locally on the tablet until the connection can be reestablished.
Web Technologist Danny Vargas admitted that creating an electronic version of the hand scored form had other advantages. “We were able to not only record scores for each assessment criteria, but also instantly calculate the scores for each section, and for the student’s overall performance. Having the individual and cumulative scores immediately available for viewing gave instructors the opportunity to review each student’s performance and provide constructive feedback on the spot.“ Students can also access the assessment database to view a secure, detailed report of their performance.
Dr. Levi had a chance to “test out’ the new testing strategy on August 13. “Overall, the process worked very well. The interface is very easy to use.” One of the issues that came up during this initial pilot phase was the quality of the tablet devices. “I’m recommending we upgrade to iPads because the quality of the display is noticeably better” said Dr. Levi. “Also, we had a timeout during one of the longer testing sessions, suggesting the need to save data periodically, much like Word can be set to autosave a document.”
Initial testing of the tablet approach has suggested future product enhancements. One of them is a built-in timer with a Start/Stop button that will track how long it takes for students to complete specific procedures. Another is automatic transfer of data to Blackboard or eValue for grade management and reporting.
The use of tablets and other mobile devices to streamline academic testing has been growing over the past year. In 2012, Blackboard Mobile Learn 3.0 introduced mobile compatible tests and ExamSoft announced the release of the first known mobile application for secure offline exam delivery on the iPad.
While these applications have created new opportunities for objective testing, the assessment of student performance in the lab and clinical setting continues to be a labor intensive process, requiring the manual entry of student scores and many hours of faculty or staff time to re-enter those scores in Blackboard, EValue, ExamSoft or other academic databases for analysis and reporting.
Last Spring the College of Veterinary Medicine successfully piloted the use of tablet devices for capturing student scores in the laboratory setting. Students equipped with an inexpensive iPad device were able to log in, rotate to different stations within the lab setting, and have their performance or test question responses electronically recorded and automatically sent to the Blackboard Grade Center. (Watch for a more detailed post on this project in October, when the Veterinary Clinical Skills course begins another round of student lab testing with a new cohort.)
For more information, see
Ohad Levi, DVM – Associate Professor, Equine Surgery can be reached at firstname.lastname@example.org