Telehealth Training Manuals and Guidelines -- trusted by medical professionals since 1985

Is Current Telephone Triage Research Disconnected?

 False Assumptions about Electronic Guidelines, Clinical Assessment & Decision Making

Research and “Real World” Practice: A Major Disconnect

Since the mid-nineties, the telephone triage industry has experienced breathtaking growth. Still, there is a lack of consensus about everything from scope of practice to terminology. For example, which title describes most accurately what clinicians do:  Telephone -advice, -triage, -consulting or telehealth, e-health, informatics, or tele-practice?

Thus, the new field endures the prodding and testing by existing disciplines, prompting the realization that the industry is operating under some false assumptions. To be sure, standards of practice have proliferated, and the industry has made phenomenal technological progress. But has the technology enhanced, driven or out paced nursing practice and standards?.

New telephone triage research, identifying several unaddressed research gaps, challenges the impression that “we have arrived.” Article titles reveal a trend toward self-scrutiny: “UCLA study finds telephone-, in person- triage of same patient don’t often agree” (1999); “Quality of telephone triage nursing comes under fire” (1999); Telephone Triage—how good are the decisions?” (1998); “Pediatric telephone triage protocols: Standardized decision-making or false sense of security?” (1999) What follows is a discussion examining current assumptions about three fundamental areas—initial assessment, critical thinking/decision-making and protocol/guidelines.

Clinical Assessment

Have we adequately defined what comprises a competent initial assessment? Over twenty years ago, Levy found that staff in an ED setting often failed to obtain the age of the patient and baseline information, frequently giving advice without obtaining adequate histories (1980). Nine years later, Verdile also found serious flaws in the practice of ED nurses (1989). A research assistant, posing as a daughter calling for her father, described a scenario that could reasonably be interpreted as an MI (when asked, the assistant said he was a 56-year-old male with chest pain, who smoked, did not drink and had no previous medical history). Among other findings, Verdile found that over half the nurses failed to ask the caller enough questions about the client or the chief complaint.

In 2001, even with the advent of formal protocols, there still seems to be little agreement about what comprises a preliminary assessment. It is confusing to new practitioners to instruct them to collect an “initial history” without providing them with some standard questions. Collecting enough of the right information initially, before consulting a protocol (assuming that no emergent symptoms are present), helps avert premature conclusions, stereotyping and “wrong train” syndrome (Clawson, 1999). This step is analogous to the brief assessment of face-to-face encounters, leading to an “initial impression,” usually formulated by physicians or nurses prior to a formal diagnostic process. In telephone triage, this step sets the stage to further explore symptoms using specific protocol(s).

In 1999, AAACN Practice standards stated that telenurses use the nursing process (assessment, planning, implementation and evaluation). Recently, they defined measurement criteria for assessment as follows “assessments based upon risk factors, need for care, and reason for call … caller’s concerns, history, and description of symptoms and assess significant data to evaluate and classify the urgency of the symptoms.” To make these standards most meaningful and measurable requires that “significant data” be further defined.

Along with age and gender, wouldn’t it be reasonable and prudent to elicit, at minimum, data regarding, 1) a problem history (chief complaint, associated symptoms and quality or characteristics, onset/duration, location and alleviating or worsening factors) and 2) a patient history (risk factors such as pregnancy, allergies, medications, and previous medical history)? I suggest we amplify and operationalize standards by creating a specific list of significant data. This will facilitate competency testing, i.e., are staff eliciting enough of the right information?

Clinical Decision Making

How do clinicians make decisions under conditions of uncertainty and urgency? How does decision-making on the phone differ from critical thinking at the bedside? In 1984, Benner first described domains of nursing expertise (coaching/teaching/diagnostic/crisis intervention/monitoring and coordinating). In 1995, Vimla Patel, expert in medical decision-making, first described the importance of decision-making expertise (comparable to Benner’s “diagnostic” domain) in telephone triage. Patel also crystallized the difficulties of the work, describing it as “decision-making under conditions of uncertainty and urgency,” and comparing it to that of air traffic controllers, fire captains and anesthesiologists.

Patel studied ED nurses’ decision-making in “real world” situations, when performing telephone triage of a researcher posing as a caller with chest pain. Nurse-participants had two to ten or more years nursing experience, and did not use protocols, but relied on their clinical judgment, experience and critical thinking skills. Among other findings, Patel’s group discovered: 1. a direct correlation between experience and decision-making accuracy (experience of 10 years or more—”experts”— increased decision accuracy) 2. nurses use a contextual approach to decision-making rather than the analytic approach reflected in physicians’ diagnostic reasoning.

Patel maintains that nurses have exceptional skill in involving the patient, and, as trained observers, are excellent at data gathering before targeting their interventions. This seminal research precipitated a flurry of articles (Crouch, 1998; Crow, 1995; Edwards, 1994; Edwards, 1998) supporting the pattern recognition approach. These findings can and should inform protocol development, training and hiring of staff.

Clinical Decision Support Guideline Standards

Are current protocols and guidelines reliable? There are now dozens of protocols and guidelines—”home grown” to nurse—and physician— developed, basic and sophisticated, in paper and/or electronic formats. These guidelines and protocols utilize two major decision-making strategies: 1. algorithms—rigid, abstract, linear, yes/no decision trees,  2. contextual, flexible “case based reasoning,” based on “classic pictures” or “pattern recognition” approaches. We still don’t know which approach is more effective, safe, user friendly or which reduces risks of mistriage.

Recently, Judith Brillman, MD, and other researchers examined the validity and reproducibility of a protocol system when they tested the consistency of nurses’ management and decision-making regarding a standardized patient (simulated call). They found little agreement in telephone triage decisions among practitioners (MDs and RNs) and protocols, concluding that use of protocols will not standardize care.

In two exchanges of letters in the Annals of Emergency Medicine in 1996 and 2000 respectively, Dr. Brillman states, “we are very much in favor of the development of well designed and reliable triage programs, the need for which our study findings report. However, where are data demonstrating that such systems exist?” And, the researchers were “not aware of a blinded, well controlled study reporting outcomes using these protocols.” Brillman’s excellent questions pose a challenge for protocol developers and users.

Experts Farnsworth and Mariani (1996) further support this query through an article describing desirable qualities for Practice Guidelines. These criteria, developed by IOM (Institute of Medicine) and AHCPR (Agency for Health Care Policy Research), apply to telephone triage as well:

  • Validity—if followed, it will lead to expected outcomes
  • Reliability/reproducibility—given the same data, another set of nurses would produce the same results
  • Clinical applicability—explicitly states the populations to which they apply
  • Flexibility—identifies exceptions to their recommendations
  • Clarity—unambiguous language, precisely defined terms, easy to follow mode of presentation.

In regard to development criteria, IOM recommends that the development process include representatives of key affected groups and have scheduled reviews. Finally, they further stress that there be documentation that sets forth the assumptions, analytic methods, rationales used in protocol development. I suggest that a general description of assumptions be provided to the end user and medical reviewer in the form of a comprehensive user’s guide, also containing detailed instructions on 1. the safe, correct operation of the protocols, 2. exceptions to any recommendations, and 3. definition of terminology.

While many developers strive to achieve high levels of the above characteristics, Brillman’s research indicates there is much work to do in the area of validity and reliability. In the area of clarity, any protocol/guideline dispositions (when/where/by whom patients should be seen) are still ambiguous. What, specifically, is meant by the words “now, at once, today, emergent, urgent, immediate, timely or contact physician/provider/office, and how does the patient interpret those words? Who is the proper agent to provide terminology for our dispositions? I believe that we (nurses) must spell out directives in lay terms with time frames that both nurse and caller mutually understand and agree upon, thus avoiding confusion and potential liability by leaving little room for interpretation.

AAACN states nurses must “participate in clinical and healthcare systems research,” adding that “nurses should evaluate and apply research findings” to practice. Now that telephone triage research and standards are more prevalent, are we making these connections?


  • American Academy of Ambulatory Care Nursing (2001). Telehealth Nursing Practice Administration and Practice Standards, Pittman. NJ: Anthony J. Jannetti, Inc.
  • Benner, P. (1984). From novice to expert: Excellence and power in clinical nursing Practice. Menlo Park, CA: Addison-Welsey.
  • Clawson, J. J. and Democoeur, K.B. (1998). Principles of Emergency Medical Dispatch (2nd Ed.) National Academy of Emergency Medical Dispatch.
  • Crouch, R. and Dale, J. (1998). Telephone triage—how good are the decisions? (Part 2), Nursing Standards, 12, 35, 33-39.
  • Crow RA, et al. (1995). The cognitive component of nursing assessment: an analysis. J Ads Nurs. Aug; 22 (2):206-212.
  • Data Strategic Benchmarks (1999). Telephone, in-person triage of same patient often doesn’t agree, UCLA study concludes. June; 3 (6): 94-5.
  • Edwards B. (1994). Telephone triage: how experienced nurses reach decisions. J Adv Nurs. Apr; 19(4): 717-724.
  • Edwards B. (1998). Seeing is believing—picture building: a key component of telephone triage. J Clin Nurs.

Written in 2001 by Sheila Wheeler Used with Permission Telephone Triage Times


Written by Sheila Wheeler (15 Posts)

SHEILA WHEELER, RN, MS, is acknowledged as an international expert in the field of telephone triage. She has practiced nursing for over 30 years, primarily in critical care, emergency department and clinic settings. An accomplished writer, educator, researcher and consultant, Ms. Wheeler is currently President of TeleTriage Systems in San Anselmo, California.