Zynx Health's mission is to provide evidence-based products and services that empower our clients to measurably improve the quality, safety, and efficiency of patient care. Zynx AmbulatoryCare is an Internet-based product that facilitates the integration of “best practices” into the workflow of primary care practitioners through our partnerships with electronic medical record vendors.
Zynx AmbulatoryCare content is divided into modules that represent clinical conditions (eg, Heart Failure – Systolic), health maintenance topics (eg, Preventive Care and Screening – Adult), symptoms (eg, Low Back Pain – Acute), or diagnostic workup (eg, Breast Mass Workup). Modules consist of 1 or more of the following components: evidence, order sets, rules, performance measure checklists, forecasting, and continuing medical education. The methodology for development of evidence, order sets, rules, and performance measure checklists is described below. The methodology for development of the forecasting component is described at: Forecasting Methodology.
EVIDENCE
The evidence pages for each module are divided into topic pages that are accessible via hyperlinks from order sets and rules.
"Zynx Health’s research experience and clinical expertise makes us confident that we’ll produce evidence-based order sets for years to come."
Leigh Miller, Director of Clinical Outcomes, AnMed Health |
Content Organization
Evidence pages are divided into the following 3 sections:
1. Reminder
- The reminder is a concise and prescriptive statement summarizing the action to be taken based on the evidence.
- There are 5 levels of reminders pertaining to diagnostic or therapeutic interventions:
- Always use
- Consider using
- No recommendation (evidence is insufficient, inconsistent, conflicting)
- Avoid routine use of
- Never use
- Each reminder level is selected based on the relative weighting of evidence concerning clinical benefit, harm, cost, and expert consensus.
2. Rationale
- The rationale section provides a summary of key messages from all cited references.
3. References
- All references are assigned a class of evidence based on the Zynx Health Evidence Classification System.
- The authors, title, and source are stated when applicable.
- For selected articles, a Synopsis or Synopsis+ (a quantitative summary of the study’s key metrics) may be written by Zynx and provided via a hyperlink.
- Hyperlinks to the following external sources are provided when they are available in the public domain: PubMed citations, relevant Web sites, and/or Adobe Acrobat Portable Document Format (PDF) files.
PubMed is provided by the National Center for Biotechnology Information (NCBI), a division of the National Library of Medicine (NLM) at the National Institutes of Health (NIH). For information on NCBI disclaimers and copyright information, see www.ncbi.nlm.nih.gov/About/disclaimer.html.
Article Identification
The primary supporting evidence for each topic is identified through searches of English-language references pertaining to human subjects, using the following sources:
- Bibliographic Databases and Systematic Reviews
- National Performance Measures
Article Selection
The eligibility for inclusion of each reference in a topic is determined based on module specificity and the degree to which each reference contributes to the overall utility of evidence for a given topic.
Module specificity for evidence is defined as either (1) a majority (ie, > 50%) of the patient population having the clinical condition or undergoing the procedure of interest or (2) the condition or procedure is stated in the title of a reference.
Articles are preferentially included in each topic page using the following citation hierarchy:
- Meta-analyses, systematic reviews, and randomized controlled trials
- Nonrandomized prospective studies
- Retrospective studies
This hierarchy is similar to the scheme used by Rhew et al (Evaluating quality indicators for patients with community-acquired pneumonia. Jt Comm J Qual Improv 2001; 27:575-590) and the BMJ Publishing Group (How clinical evidence is put together: searching and appraising the literature. Clinical Evidence 2001; 6:xviii). The following criteria are also applied:
- More recent publications are preferentially cited over older publications.
- For a given topic, the rationale section may be divided into sections in which the evidence pertaining to each section follows the same citation hierarchy.
- When there are different studies of the same class of evidence for a given topic, studies that describe clinical outcomes (eg, mortality rates, quality of life) are preferentially cited over studies that describe nonclinical outcomes (eg, laboratory values, radiographic findings, statistical techniques).
- When there is conflicting evidence, articles of varying levels of evidence may be included to address the topic more comprehensively.
- Guidelines and consensus statements from national organizations are cited for each topic when available.
Topic Selection: Adult Modules
Topic pages are created for processes of care based on available evidence. Topics for each module are chosen based on any of the following criterion groups:
- Class P Criteria (National Performance Measures)
- The topic is addressed as a national performance measure.
- Class E Criteria (Guidelines)
- A reference with recommendations from a national organization (but not an individual author) is published in the form of a guideline, consensus statement, position statement, or technology assessment.
- References with recommendations from national organizations are eligible for inclusion if they are published in journals that are indexed in PubMed. References that are not indexed in PubMed may be cited if an older version of the guideline from the same organization was previously indexed in PubMed.
- Class M and S Criteria (Meta-Analyses and Systematic Reviews)
- A meta-analysis demonstrates statistically significant outcome for a clinical endpoint between an intervention group and a comparison group.
- A meta-analysis demonstrates equivalency for a clinical endpoint when an intervention is compared to an alternative intervention that has been shown to be superior to a control group.
- A systematic review addresses a clinical endpoint.
- Meta-analyses and systematic reviews may be used to define new topics irrespective of whether their results demonstrate benefit, harm, or no effect.
- Class A Criteria (Randomized Controlled Trials)
- At least 2 randomized controlled trials demonstrate a statistically significant outcome for a clinical endpoint between an intervention group and a comparison group.
- At least 2 randomized controlled trials demonstrate equivalency for a clinical outcome when an intervention is compared to an alternative intervention that has been shown to be superior to a control group.
- A single randomized controlled trial involving > 1,000 patients demonstrates a statistically significant outcome for a clinical endpoint between an intervention group and a comparison group.
- Class B and C Criteria (Nonrandomized Prospective and Retrospective Studies)
- A single nonrandomized study involving > 10,000 patients demonstrates a statistically significant outcome for a clinical endpoint.
- Class Q Criteria (Economic Analyses)
- At least 2 studies demonstrate an economic advantage or disadvantage for a process of care.
- Diagnostic Article Criteria
- An article about a diagnostic test defines a reference standard for comparison, is available for use in a clinical setting, and reports any of the following test characteristics: area under the receiver operator characteristics curve, positive or negative likelihood ratio, sensitivity, or specificity.
- Prognostic Article Criteria
- Scoring systems, prediction rules, and other studies assessing multi-factorial clinical prognostic tools may be included if they are prospectively validated.
Processes of care are excluded when they are not available for use in clinical practice (eg, medications, laboratory studies, and diagnostic tests that are not commercially available).
Topic Selection: Pediatric Modules
For pediatric modules, a topic page may be created if any of the following criterion groups are satisfied:
- Class P Criteria (National Performance Measures): same criteria as for adult medical modules
- Class E Criteria (Guidelines): same criteria as for adult medical modules
- Class M and S Criteria (Meta-Analyses and Systematic Reviews): same criteria as for adult medical modules
- Class A or B Criteria (Randomized or Nonrandomized Prospective Studies)
- A prospective clinical trial addresses a process of care (eg, an intervention, test, or prognostic factor) that can be applied in current clinical practice (ie, it is not limited to experimental or research applications).
- Class C Criteria (Retrospective Studies)
- A single retrospective study involving > 1,000 patients demonstrates a statistically significant outcome for a clinical endpoint.
- Class Q Criteria (Economic Analyses)
- Any study that demonstrates an economic advantage or disadvantage for a process of care.
- Diagnostic Article Criteria
- An article about a diagnostic test defines a reference standard for comparison, is available for use in a clinical setting, and reports any of the following test characteristics: area under the receiver operator characteristics curve, positive or negative likelihood ratio, sensitivity, or specificity.
- Prognostic Article Criteria
- Scoring systems, prediction rules, and other studies assessing multi-factorial clinical prognostic tools may be included if they are prospectively validated.
Processes of care are excluded when they are not available for use in clinical practice (eg, medications, laboratory studies, and diagnostic tests that are not commercially available).
ORDER SETS
Zynx default order sets are composed of order items and reminder items that are grouped into sections.
Order Items
Order items in Zynx default order sets may be evidence-based, in which case an “Evidence” hyperlink is provided. Additional common practice order items may also be added for completeness. National performance measures are denoted by blue ribbon icons with pop-up text descriptions indicating each performance measure's source.
Medication order items consist of a drug name as well as 1 or more permutations of dose, route, frequency, and other attributes. Medication order items may be evidence-based or common practice and are not necessarily approved by the U.S. Food and Drug Administration for use in the respective clinical module. Recommendations by the Institute for Safe Medication Practices for abbreviations and dose designations are reviewed and applied to the order sets in a timely manner (http://www.ismp.org/Tools/errorproneabbreviations.pdf). When applicable, look-alike and sound-alike medication names are displayed using “tall man” lettering in compliance with the U.S. Food and Drug Administration Name Differentiation Project (www.fda.gov/cder/drug/MedErrors/nameDiff.htm).
Reminder Items
In some cases, clinical evidence may satisfy topic inclusion criteria but cannot be directly translated into order items, so a reminder item is embedded in the order set, and a hyperlink to the evidence page is provided. For example, a topic regarding the use of a prediction rule to guide triage decisions for patients with community-acquired pneumonia cannot be translated into an order item, but an order set reminder item can serve as a timely evidence-based reminder to the physician at the point of care. Reminder items are created only for topics that are presented in the evidence. National performance measures are denoted by blue ribbon icons with pop-up text descriptions indicating each performance measure's source.
Sections
Order set section names are internally standardized healthcare concepts that are used to group order items into the following top-level sections: Condition, Vital Signs, Activity, Nursing Orders, Respiratory, Diet, IV Fluids, Medications, Laboratory, Diagnostic Tests, Specialty, Consults, and Noncategorized. Sections are often divided into subsections for the purpose of subgrouping similar concepts. The ontology of sections, order items, and reminder items within Zynx default order sets can be rearranged and renamed during customization in the AuthorSpace environment.
Zynx Medication Quality Assurance (QA) Process
For publication of new or updated medication order sentences in Zynx published content, 3 Zynx clinicians (consisting of at least 1 pharmacist) independently review the medication order sentences based on information contained in drug textbooks, handbooks, databases, or other third-party sources.
Zynx may make edits to medication sentences in the Zynx published content based on such review. Zynx reserves the right to make such edits in the client’s customized content located on Zynx servers, including any content in released status, and notify the client of such edits.
RULES
Sample rules are provided to exemplify how evidence can be translated into alerts and reminders within electronic medical record systems.
Rule Organization
Zynx sample rules are written for various clinical scenarios that may apply to a single venue of care (eg, an admission or discharge reminder) or multiple venues of care (eg, an alert to avoid a medication). Rules are written in plain English with Boolean operators (eg, “AND”, “OR”, “NOT”). Each rule contains 3 main sections: Evoke, Logic, and Action.
The Evoke section describes criteria that could potentially trigger the rule.
The Logic section provides clinical criteria that should be satisfied for 1 or more actions to be performed. Not all evoking events or logic criteria may be available for all electronic medical record systems, although surrogate markers may be substituted at the discretion of each organization.
The Action section provides sample alerts and reminders with possible actions that could be performed as a result. In clinical scenarios where multiple logic criteria may be satisfied in more than 1 permutation, subsets of logic statements are conceptually related to each other using action groups. In these instances, the logic criteria are listed as L1, L2, L3, etc. and the action groups are listed as A1, A2, A3, etc. Items in the Action section may be hyperlinked to corresponding Zynx evidence.
Additionally, the purpose of the rule is displayed.
Rule Selection
Clinical rules are derived from at least 1 of the following criteria:
- Topics that include at least 1 Class P article
- Topics that include at least 1 Class M or Class S article
- Topics that include at least 2 Class A articles
- Topics that include at least 2 Class E articles
Clinical rules are developed and reviewed by Zynx clinicians if the clinical evidence topic can be translated into the appropriate syntax.
PERFORMANCE MEASURE CHECKLISTS
For selected Zynx AmbulatoryCare modules, performance measure checklists are available as PDF or Microsoft® Word files that detail important processes of care, as well as their expected frequencies and goals, as defined by Class P articles. Data for each of these processes of care can be completed for an individual patient at the point of care or as part of a retrospective chart review.
PRODUCTION ISSUES
Editorial Process/Quality Control
All components of Zynx AmbulatoryCare are written and maintained by Zynx physicians. Additionally, a Zynx librarian assists with literature searches and reference cataloging. Zynx pharmacists review medication-related order sentences and their corresponding evidence. Zynx editors review all content for technical accuracy and style consistency. External clinician review is also performed for selected modules.
Physician Editorial Advisory Board
The Zynx Health Physician Editorial Advisory Board is comprised of accomplished and respected physicians who represent a wide spectrum of medical disciplines and specialty areas.
Update Process
Zynx AmbulatoryCare content is updated every 6 months; additional interim updates may be performed on selected modules based upon the emergence of new evidence. Hyperlinks to external Web sites and documents are reviewed on a regular basis to verify functionality.
Forecasting Methodology
INTRODUCTION
Zynx Forecasting modules have been developed to complement the evidence-based literature in ZynxEvidence or Zynx AmbulatoryCare by providing a tool with which to calculate the potential benefits that may be realized through implementation of any of the "best practices" presented there.
Zynx Health employs a systematic methodology that includes evaluating the overall body of evidence in selecting topics and supporting articles for inclusion in Zynx Forecasting modules. It should be noted, however, that subjective judgments are sometimes required. All such judgments are made by physicians with expertise in the principles and practices of evidence-based medicine and in critical appraisal of the literature.
SELECTION PROCESSES
Process for selecting topics to be used in Forecasting
The Zynx Health clinical content development team performs a comprehensive evaluation of the overall body of evidence to determine whether an intervention results in benefit or harm, or if the evidence is conflicting or inconclusive. If the overall evidence is inconclusive or conflicting, this intervention is not a Forecaster candidate.
Process for selecting studies to be used in Forecasting
- Only those candidate studies whose findings are consistent with the overall body of evidence regarding the intervention are identified as eligible for the Forecaster.
- From the pool of eligible studies identified in step #1, those studies whose findings are "modelable" (see section below on modelability) are identified.
- From the pool of studies that are eligible and modelable as defined in step #2, those studies that fulfill the following criteria are identified:
- Are applicable to the majority of patients with the condition or undergoing the procedure (for Key Aspects of Care; not necessarily for Cost)
- Have a demonstrated impact on clinical outcomes and/or costs
- Have the potential to improve care and/or provide cost benefits by narrowing "utilization gaps" (eg, inappropriate under-utilization, inappropriate over-utilization)
- Are a focus of specific Zynx Health recommendations targeted at improving the rate of appropriate utilization and/or reducing practice variation
- Preference is given to larger studies and to those studies with better methodology in their study design. The latter preference is based on the following hierarchy of study designs:
- Good: retrospective studies (Class C studies)
- Better: nonrandomized prospective studies (Class B studies)
- Best: randomized controlled trials (Class A studies) or meta-analyses (Class M studies)
MODELABILITY
A study that addresses a process of care is defined as "modelable" when a mathematical model can be constructed from the evidence presented in that study, and when that model can be used for forecasting the benefits (or harm) of implementing a given process of care. The corresponding process of care is then also defined as modelable. Requirements that must be satisfied in order for a study to be modelable include:
- The processes or interventions in the study are associated with a statistically significant outcome or outcomes.
- The processes or interventions in the study are associated with clinically significant or financially important benefits.
- The independent and dependent variables in the study are dichotomous when used to model Key Aspects of Care.
- A mathematical model describing the benefits of implementing the process can be developed and defined.
- No major conflicts are found among the studies addressing the process.
"NON-MODELABLE" MODULES
A ZynxEvidence or Zynx AmbulatoryCare clinical module may occasionally be found for which there exists no modelable Key Aspect of Care or Cost. In this case, the module is defined as "non-modelable." Each update cycle, the new literature is examined to determine whether a Forecaster can be developed for each previously non-modelable module. Appendix 2 lists modules currently deemed non-modelable.
SELECTION OF SUPPORTING STUDIES FOR FORECASTING
Literature Identification
Material for inclusion in Zynx Forecasting modules has been selected primarily on the basis of a systematic review of the biomedical literature indexed in online bibliographic databases and other sources. Sources include:
Searches encompass the ten years immediately preceding the year of the search; in some cases, landmark articles are included from an earlier time frame.
- Primary search inclusion criteria
Meta-analyses; systematic reviews; randomized controlled trials; nonrandomized prospective studies; retrospective studies; decision-analyses; performance measures; overviews; and guidelines are included.
- Primary search exclusion criteria
Non–English-language articles, letters, and editorials are excluded.
LITERATURE SCREENING AND ACQUISITION
Titles and abstracts (where available) of articles retrieved by the online searches are reviewed by physicians. Those articles which appear to be candidates for providing relevant topic information are retrieved from the collections of a major university biomedical library and a major medical center medical library (articles from periodicals not held by these libraries and not available to the editorial team from other collections are not considered).
LITERATURE CLASSIFICATION
Articles retrieved for review are assigned a class of evidence based on the Zynx Health Evidence Classification System.
SUPPLEMENTAL LITERATURE SECTION
The Supplemental Literature page featured for each Key Aspects of Care or Cost is derived from the corresponding page in ZynxEvidence or Zynx AmbulatoryCare. It may include references or synopses in addition to those that may or may not include quantitative information or have been used specifically for modeling the Forecaster. The Forecaster Guideline or Cost-Savings Opportunity recommendation may not be identical to the Reminder on the Supplemental Literature page, since the Forecaster may suggest a more specific intervention than the more general statement in the Reminder.
FORMULAS
Applying Results Drawn from Articles to Forecasting Calculations
The formulas presented on the Sample Calculations pages (and also in Appendix 1) are used to generate the Forecasting Calculator outputs (on the corresponding Forecasting Calculator page, or summarized on the Calculations Summary page). Parameters defining the statistics of the intervention, eg, the relative risks and absolute risk reductions, are extracted or calculated from peer-reviewed studies.
Assumptions
In some cases, assumptions are made to facilitate the development of the mathematical model. For example, data from the study may be used to calculate a potential benefit for a population of patients not specifically addressed by the study (eg, extrapolating the benefit of pneumococcal immunization to the entire population of community-acquired pneumonia patients, rather than the subset for which benefit was demonstrated in the study). In such cases, the assumptions are discussed in the Limitations section.
Selection of Default Values
Although organization-specific data should be input to the Forecaster when available, default values are provided for use when organization-specific data are not available.
When possible, the default values are based on information from the peer-reviewed literature and databases in the public domain. In cases where more than 1 article provides possible default values, the selection of default values is based on the following factors:
- Generalizability of the study population based on patient demographics and risk levels, multicenter versus single-site, and US versus non-US
- Size of the study
- More recent data
In cases where default values cannot be obtained from the peer-reviewed literature or public databases, then unpublished reports, conference abstracts, and other sources of data may be used. When these sources of data are unavailable for a specific default value, Zynx Health physicians provide an estimate for the parameter, taking care to be conservative, so that the final calculated results are likely to underestimate the actual potential benefits.
The resulting default values are then presented and referenced (if a reference to the peer-reviewed literature exists) in a table on the Sample Calculations page.
DEFINING CONTRAINDICATIONS TO MEDICATIONS
Contraindications to medications and other interventions may vary in the literature and in published guidelines. The following methodology for defining the contraindications presented on the Definitions pages of the Zynx Forecasting modules applies:
- In cases in which the study used to model the Forecaster is a clinical study, the contraindications will be those specified in the study, if provided, or by a regulatory organization. In cases in which the study is a meta-analysis, the contraindications presented will be those published by a regulatory organization, where possible.
- If a regulatory organization does not provide a list of detailed contraindications for the specific intervention, then the list of contraindications will be based on those presented in guidelines published by a specialty society, when available.
- When contraindications are not available from literature published by a regulatory organization or a specialty society, then other sources will be used and referenced.
EDITORIAL PROCESS/QUALITY CONTROL
Zynx Health physicians initially select the material for inclusion in Zynx Forecasting modules. Article synopses are presented in many cases. These have been written either by Zynx Health clinicians or by other professional medical writers. Hyperlinks to abstracts in PubMed, when available, are provided in the Supplemental Literature pages. Content is edited for technical accuracy and copyedited for spelling, grammar, and readability by at least 1 professional editor. Zynx Health clinicians write all summary statements, and check for completeness and accuracy. All literature summaries and data used for calculations are verified by a physician reviewer.
2 x 2 table:
| |
Outcome Present |
Outcome Absent |
|
| Treatment |
a |
b |
(a + b) |
| Control |
c |
d |
(c+ d) |
| |
(a + c) |
(b + d) |
(a + b + c + d) |
Definitions for statistics:
|
ABI |
Absolute Benefit Increase
= | EER – CER | |
|
ABR |
Absolute Benefit Reduction
= | EER – CER | |
|
AOR |
Adjusted Odds Ratio |
|
ARI |
Absolute Risk Increase
= | EER – CER | |
|
ARR |
Absolute Risk Reduction
= | ( a / [a + b]) – (c / [c + d]) |
= | EER – CER | |
|
CER |
Control Event Rate
= c / (c + d) |
|
CI |
Confidence Interval |
|
EER |
Experimental (Treatment) Event Rate
= a / (a + b) |
|
HR |
Hazard Ratio |
|
LR(-) |
Likelihood Ratio for negative test |
|
LR(+) |
Likelihood Ratio for positive test |
|
NNH |
Number Needed to Harm
= 1 / ARI |
|
NNT |
Number Needed to Treat
= 1 / ARR |
|
OR |
Odds Ratio
= (a / b) / (c / d)
= ad / bc |
|
ROC |
Receiver Operating Characteristic |
|
RR |
Relative Risk
= EER / CER
= (a / [a + b]) / (c / [c + d])
= (a * [c + d]) / (c * [a + b]) |
|
RRR |
Relative Risk Reduction
= 1 – RR |
|
SD |
Standard Deviation |
|
W |
Number of patients admitted per year with the condition |
|
X |
Percentage (%) patients already receiving the treatment |
|
Y |
Percentage (%) patients for whom the treatment is contraindicated |
|
Z |
Event rate for patients in the comparison (usually control) group |
Converting between statistics:
If OR is known:
Number of events averted =
W * ([100 – X – Y] / 100) * (1 – [OR / [{1 – Z} + {OR * Z}])* Z
If RR is known:
Number of events averted = W * ([100 – X – Y] / 100) * (1 – RR) * Z
If RRR is known:
Number of events averted = W * ([100 – X – Y] / 100) * (RRR)* Z
If ARR is known:
Number of events averted = W * ([100 – X – Y] / 100) * ARR
REFERENCES
- Evidence-based Medicine: How to Practice and Teach EBM. 1st edition. David L. Sackett et al. Secaucus, NJ: Churchill Livingstone; 1997.
- Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature, II: how to use an article about therapy or prevention, A: are the results of the study valid? JAMA 1993; 270:2598-2601.
- Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature, II: how to use an article about therapy or prevention, B: what were the results and will they help me in caring for my patients? JAMA 1994; 271:59-63.
- Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Controlled Clinical Trials 1996; 17:1-12.
- Greengold NL, Weingarten SR. Developing evidence-based practice guidelines and clinical pathways: the experience at the local hospital level. Joint Commission Journal Quality Improvement 1996; 22:391-402.
Modules currently found to be non-modelable are:
- Acute Pain
- Acute Sickle Cell Crisis – Adult
- Cancer – Lung, Small Cell
- Cancer – Prostate
- Cystic Fibrosis – Pediatric
- Fever Without a Source – Pediatric
- Gastrointestinal Bleeding – Lower
- Hypoglycemia – Neonatal
- Nephrectomy
- Sepsis – Neonatal
- Sepsis – Pediatric
- Symptoms