ChatGPT has become an increasingly popular tool with many uses. It is very useful for many purposes, but it is very important for cat owners to know that ChatGPT is not a doctor (although it may play one on TV, so to speak!)

(Just to be clear, the points made in this article apply to all publicly available AI programs, not just ChatGPT.)

As with all computer programs (and that really is all ChatGPT boils down to), the GIGO rule applies: Garbage In, Garbage Out.

There are two categories of what one might consider “garbage” that go into the ChatGPT functionality.

The first is the “information” that ChatGPT is getting from the Internet as a whole. Much of what is published on the Internet is outdated, personal opinion (with or without scientific support), influenced by politics or other belief systems, or just plain wrong. ChatGPT has no way to truly judge the quality of the information that is being fed into it. It works using statistical analysis of a massive dataset of word counts and combinations. The fallacy here is that the Internet is a giant echo chamber in which opinions, outdated ideas, and so on are repeated again and again. Basically, the more often a concept is repeated on the Internet, the more likely that ChatGPT is to use that concept as an assumed truth. This dynamic is more likely as a topic becomes more complex or nuanced.

Medicine is a very complex and nuanced field. It is a natural human tendency to use reductive thinking to apply to one’s own immediate situation, but when proper healthcare is on the line, it is wise to resist this reductive thinking pattern, and to seek the opinion of an actual expert.

The second source of “Garbage In“ when using ChatGPT falls into the category of Prompt Engineering, which is becoming an entire field of study in and of itself. Basically, what “prompt engineering” means is that you have to ask exactly the right question to get a useful answer.

ChatGPT does not help with asking the right question. It takes any question at face value, including any and all assumptions made in that question.

A good example of “assumptions contained in a question” would be the question: “how should I treat my cat’s eye infection?” The underlying assumptions are:

  • an assumption that the symptoms exhibited by the cat are in fact caused by an eye infection, and not something else, such as a corneal laceration;
  • the fact that “infection” is implicitly understood by ChatGPT as “bacterial,” whereas nearly all feline “eye infections” are actually herpetic (ie: viral) in nature;
  • an assumption that there are no other relevant concerns, such as other symptoms or historical / underlying problems that may be related;
  • an assumption that the eye problem, whatever it is, should be “treated,” when in fact it may well resolve on its own.

Chat GPT will not “question the question,” which is certainly the first and possibly the most important thing that a doctor does. ChatGPT does not “take a history.”

Early in the history taking process, a doctor will untangle patient theories (“I have the flu”) from actual facts (“I have a fever and chills”), and will ask further questions to elucidate what may be happening. The taking of a good history is an important and indispensable skill every doctor must have, and a complete history must be taken even to begin to establish the list of possible diagnoses for a given problem, including a review of all historical medical problems the patient has had. Every patient’s life is a story, and the consideration of various possible diagnoses depends crucially on how that narrative has unfolded. Dr. Josh Tamayo-Sarver, a human ER doctor with a Ph.D in Epidemiology and Biostatistics, also writing about the applications of ChatGPT to the practice of medicine, points out that incorrect diagnoses are often fatal, and adds: “The art of medicine is extracting all the necessary information required to create the right narrative.” This is not as easy as it sounds, and ChatGPT does not do this at all.

Further, once the doctor has taken the history, the doctor must apply to this information his or her full medical education, plus experience, to guide him or her through this rigorous process, in order to arrive at a credible list of possible diagnoses (“differential diagnosis”). ChatGPT does not ask any questions; it simply takes the diagnosis assumed in the question as correct, although the questioner may not be a doctor. The taking of a good history is a key part of reaching a good diagnosis, and taking that history requires a full medical education in order to know what questions to ask. (I would go further to say that, at least in feline medicine, the process of taking a full and discerning history is nearly always far more important and revealing than the physical examination – The Author)

Another salient point here is that clients and patients are not trained observers; they are not trained or experienced in deducing the meanings of their observations from a medical standpoint. It is not only common, but nearly universal, that a client with a sick kitty will have seen something at home that is very important, but the client didn’t know that it was important and so didn’t report it – until asked by the doctor. When a client or patient relies on ChatGPT, they are nearly always supplying incomplete information, because even though they may have that information, the client / patient doesn’t realize the significance of it. And neither does ChatGPT.

Another important concept in the diagnostic process is the importance of body language and tone of voice in history taking. These things comprise more of communication than words do. Doctors are experienced at assessing these things, and judging the utility of the words they are hearing based on the body language. As Dr. House repeatedly stated, “everybody lies,” but that’s not the only reason that non-verbal cues are important. People are naturally reluctant, often subconsciously, to tell the doctor things that they feel they may be “judged” about, and such things are not uncommonly key to the diagnosis. This communication element is entirely missing from ChatGPT.

Medical intuition is another important tool your doctor brings to the diagnostic process. A doctor’s mind is overflowing, not just with education, but with all the experiences of multiple patients and cases; it is essentially a giant database. “Intuition” is just another word for what comes to the top of the mind when some pattern is subconsciously recognized. Medical intuition is most important in unusual cases, where the problem or the presentation is not typical, common, or standard. Since ChatGPT is, by definition, supported by what is reported the most by the most people, it cannot have a “medical intuition” to help recognize unusual causes of problems. It is therefore likely to miss the most important possible differential diagnoses, simply because they are uncommon.

Here is an example of a human asking ChatGPT about an “eye infection”:

Kitty owners who witness ocular symptoms may panic and desire immediate intervention – that’s just human nature! – and ask for antibiotics based on a ChatGPT interaction such as the above – even though nearly all feline ocular symptoms are viral in origin, and antibiotics won’t help at all in these situations. Any number of incorrect treatments may actually prolong or worsen the problem. Unnecessary use of antibiotics leads to increased antibiotic resistance, a serious problem that kills over 35,000 people every year in the US alone. The symptoms seen by the owner may have a different cause; if so, taking active steps based on this assumed diagnosis may at best delay proper diagnosis and treatment, and at worst may cause injury. Injury caused by delay of proper treatment, or by improper treatment, may be irreversible.

In this same situation, a doctor would know to ask the right questions to elucidate information leading to a correct diagnosis and treatment that is both timely and proper.

Even though ChatGPT recommends talking to a healthcare professional, it provides a list of antibiotics anyway. This fundamentally ignores real human behavior, and might lead our kitty owner to raid the medicine cabinet and use old, expired medications. Nowhere in this answer does it make clear to the reader that the symptoms they are seeing may be caused, not by an “eye infection,” but by something else entirely. A doctor would know to explain this if asked the same question. And nowhere in this answer does it make clear that diagnostic tests may be indicated, and that “treatment” as such may or may not be indicated. Again, a doctor would know to explain that if asked the same question.

It may also be worth considering that medical doctors conform to good practice standards, hopefully for ethical reasons, but also in part by the concerns of Board complaints and lawsuits. Doctors carry insurance for this purpose, because doctors are in fact liable should they be convicted of malpractice or negligence. ChatGPT cannot be sued, nor is there any standard it is held to, nor does anyone anywhere carry any liability for the answers Chat GPT provides. (Open AI, the developer of ChatGPT, is thought to perhaps carry some liability here, but this has not yet been tested.) ChatGPT does not have an ethical framework either. In other words, there is no supportive ethical or legal infrastructure that can hold ChatGPT responsible for the results of these answers, or to help assure you that answers provided by ChatGPT are even close to correct.

Another concept to consider here is some old common sense advice we’ve all heard at some time or another: that anything after the word “but” in a sentence should often be ignored. In this instance, when ChatGPT says “I’m not a doctor, but…”, it is wise to ignore everything after the word “but.” ChatGPT is not a doctor. Period.

An excellent and fascinating article by Dr. Rachel Draelos about the comparative “diagnostic skills” of medical doctors and ChatGPT can be found here. (Spoiler Alert: doctors can diagnose. ChatGPT cannot do so by itself.)

A final concern about ChatGPT is the effect that it is having on medical practice costs, and hence the cost of medical care to the patient or client. A doctor’s only salable value lies in their knowledge and experience, and the cumulative time it takes to share that knowledge with a patient or client. More and more patients and clients are confronting doctors with the “information” they have gleaned from a ChatGPT search, rather than simply self-reporting symptoms. The extra time required by doctors to discuss the ChatGPT output is “billable” time; in other words, whether or not the patient / client pays that cost directly, overall medical care costs will increase because of the time that is now being asked of the doctor to assess, and usually debunk, whatever ChatGPT has spit out. The irony here is clear: most people who are using ChatGPT for this purpose are doing so, at least in part, in order to decrease their medical costs, but it does not work out that way.

You may also want to consider that the act of bringing ChatGPT output to your doctor may not endear you to that doctor. If you want the doctor to have good feelings about you, consider not doing that. Asking your doctor to address what you’ve gotten from ChatGPT wastes the doctor’s time and exhibits a fundamental lack of respect for the profession and for the doctor as a professional expert. If you need a second opinion, get one from another doctor, not from a computer program. While you might find ChatGPT interesting, a doctor does not need to be educated by an internet chatbot. Their degree proves that.

It seems very likely that the learning abilities of ChatGPT and similar AI models will indeed be medically useful at some point in the future; some are useful now, but in the hands of professionals only. Publicly available chatbots were not designed to be used in this way, and are absolutely not “ready for prime time” insofar as being able to provide consumers with useful, actionable medical information. You undoubtedly care about your own health and that of your cat very much. We advise you to exercise good judgment and healthy skepticism when using shiny new, unproven (and very often wrong) toys in an attempt to replace your doctor’s proven care.

Sincere thanks to Dr. Rachel Draelos , MD, PhD, for permission to use information and graphics from her articles. Dr. Draelos is a physician, and also holds both a M.S. and a Ph.D. in computer science from Duke University. Her research focus is Artificial Intelligence. Her background makes her uniquely qualified to assess both the benefits and flaws attaching to the use of AI in the practice of medicine, and the use of chatbots such as ChatGPT by non-doctors in an attempt to diagnose or treat their own problems. Her excellent and expert articles can be found at glassboxmedicine.com.

Messenger SMS Print
Tags:
Posted in: Education
Previous: Shine Pet Food Avian Flu Update