GENEVA - If you really want to find out how well PACS is working at your facility, you can follow your heart, the advice of experts, or the data, as researchers from the Netherlands have done. They discussed their work on quantifying the factors that make PACS useful at the Computer Assisted Radiology and Surgery (CARS) meeting.
Rogier van de Wetering and colleagues began their massive project with a top-to-bottom review of every PACS performance study on the books, followed by a thorough analysis of the factors that made the biggest difference in creating the "mature" information system that does more and works better than other systems. Along the way, they've been surprised by some of the answers.
In a pair of CARS presentations on Thursday, van de Wetering told how he and Ronald Batenburg, PhD, tore apart nearly 1,000 PACS performance studies to determine the factors that most starkly improve performance and utility.
Previous studies have indicated a need for broad evaluation perspectives and holistic approaches to PACS performance and performance measurement from different perspectives, said van de Wetering, who is from University of Utrecht's department of computer sciences. Unfortunately, he said, no recognized work defines what PACS performance is, let alone what it should be.
"If you look at literature today, you see many PACS evaluation studies that look at single issues, and do not portray a holistic view of what performance actually is," van de Wetering said.
In other words, he said, there are a lot of studies that look at how a facility fared after replacing its PACS, or revising some component of the operation, but none that examine what PACS performance is and what makes it better. In their CARS presentation, he discussed how they tried to fill this void.
What makes PACS useful?
The first study that van de Wetering and Batenburg conducted reviewed the PACS literature and applied a meta-analytic approach to PACS performance -- determining, in effect, what elements make a PACS network useful. Their second study analyzed the factors that drive PACS and information systems acquisition and created a framework by which facilities can plan the level of PACS functionality -- or PACS maturity, in the language of the researchers.
In total, the group retrieved 980 PACS performance papers from data repositories such as PubMed and Scholar, including 37 key publications and position papers published from 2000 to 2009.
The literature sources used a variety of qualitative and quantitative methodologies, including some that focused on diagnostic and clinical elements, and others that focused on things such as productivity and workflow efficiency, or service functions.
From these, the group came up with four principal perspectives for looking at performance issues in PACS:
- Clinical contribution (interpretation agility, efficacy of PACS, contribution to clinical communication and collaboration, interpretation time, diagnostic accuracy, etc.)
- Organizational efficiency (timeliness, cost-effectiveness, number of exams, budget ratio, etc.)
- Service (perceived service to patients, referring physicians, and users; patient flow and throughput; patient, physician, and user satisfaction; delivery times; etc.)
- Technical information system (speed of display, availability of old studies, average display time, etc.)
"We tried to construct a framework from which we could look at PACS performance from a broad perspective, and in the end we wanted to measure this at several sites and have done so," van de Wetering said. "These measures ... don't only look at what it's like to go from film-based to digital operation; you can also use it if you already have a PACS and want to measure its efficiency."
PACS maturity
The second study, based on the underpinnings of the first, as well as on a large survey of hospitals in Europe, sought to analyze and describe the elements of maturity of PACS and other information systems in a medical facility.
In this study, van de Wetering, Batenburg, and co-investigator Reeva Lederman, PhD, from the University of Melbourne in Australia, performed a focused analysis of the factors that influenced the adoption of PACS and related medical information systems and information technology, developing a quantitative scale for measuring the usefulness of PACS at a particular institution.
The study had the following goals:
- To construct a medical information systems maturity scale (MISIS)
- To identify the key determinants of a hospitals' score on MISIS
- To provide general guidelines for PACS maturity and improvement
In all, 834 hospitals in 18 different European countries were included, based on a database compiled in 2006 by E-Business W@tch, an organization that studies the impact of information technology on industries and the broader economy.
"Although the survey is from 2006, we believe it still provides some interesting perspectives," van de Wetering said. The survey included 344 hospitals with 100 or fewer beds, 226 hospitals with 101 to 250 beds, 155 hospitals with 251 to 750 beds, and 49 hospitals with more than 750 beds.
From the survey, the investigators extracted the questions that identified the key components of PACS functionality in a hospital.
Those questions queried hospitals for their use of difference types of healthcare informatics technology, such as RIS, PACS, patient administration systems, pharmacy management systems, electronic prescription transmission, and computerized physician order-entry (CPOE) systems.
Then the frequency of negative or affirmative responses to these questions was analyzed, with the results used to create the MISIS scale for hospitals.
A conceptual model for explaining a facility's PACS maturity, or level of utility, was constructed using multivariate regression analysis. The results can be used to develop conceptual guidelines for hospitals to improve and assess their PACS functionality and ambitions for future functionality, van de Wetering said.
One way the survey data showed its age was that just 32.4% of surveyed hospitals used PACS exclusively in 2006, van de Wetering said. Finnish and Norwegian hospitals had widely adopted PACS (at more than 65%) by 2006, while French and Polish hospitals lagged behind at around 10%.
PACS was most commonly used in combination with:
- Patient administration systems
- Radiology information systems
- CPOE systems
- Pharmacy management systems
Important drivers of the score included the perceived positive attitude toward IS/IT impact (e.g., efficiency, work, quality), as well as a facility's adoption of the HL7 standard, the information systems' impact on the overall business functions (management, R&D, logistics, etc.), investments in IT training, and growth as expressed in turnover and increases in staffing levels.
Not significant in this study were aspects of the workplace such as organizational structure, tasks and job descriptions, or education and training of employees, van de Wetering said. Nor did the type of hospital have any effect on the MISIS score, or whether a facility had outsourced its IT services.
 |
| Click here to enlarge this image. |
| Based on MISIS criteria, the researchers developed a scale of PACS/IS maturity that organizations can use to assess where they are in terms of information systems maturity and chart where they want to go. Image courtesy of Rogier van de Wetering. |
"These results appear to be stable over countries, despite the finding that countries within Europe do strongly differ in PACS use and their MISIS level," he said, adding that PACS and medical IS/IT maturity are situational.
Based on the results, the group went on to create guidelines for improving PACS and related medical IS/IT maturity, applying the concepts of strategic alignment -- making the information systems fit the facility's needs and ambitions -- and situational growth, which provides guidance for the steps a hospital should take when it wishes to "mature" its PACS system by growing into new levels of functionality.
"We see that hospitals with a high score on MISIS perceived more added value from their IT systems, van de Wetering said.
An important advantage of the system is giving hospitals a way to chart where they want to go, and a blueprint for how they can respond to competitive pressures and focus their strategic resources, he said.
"I believe the desires of an organization about where it wants to be ... cannot be achieved without actively governing this process, and focusing on which maturity level it wants to go to," van de Wetering said.
The work has recently been published in the online version of the International Journal of Computer Assisted Radiology and Surgery (published April 10 before print).
For their next act, the scientists have taken on the work of validating each of the MISIS criteria in a survey of Dutch hospitals, 60% of which have already completed the survey, van de Wetering said. He envisions expanding the project to hospitals across Europe and possibly the U.S. to develop a comprehensive, worldwide PACS utility blueprint.
By Eric Barnes
AuntMinnie.com staff writer
June 25, 2010
Related Reading
Is the iPad radiology's dream device for mobile healthcare? June 24, 2010
PACS, advanced visualization integration continues to evolve, June 17, 2010
New software facilitates mining of image data, May 24, 2010
Radiologists may be prone to work-related injuries, May 3, 2010
Top 10 considerations for FFDM and PACS integration, October 4, 2007
Copyright © 2010 AuntMinnie.com
