From a friend who is a doctor, we received the announcement of an event about the use of ChatGPT in Health to be held by the Faculty of Medicine of the University of São Paulo (USP) in March 2023.

In the conversation, he challenged the app by asking it who I was. The answer came with some correct information but a lot of basic information was wrong, including my medical background.

Then, we made a search for the subject of syphilis diagnosis. Again, some hits. However, elementary facts were not cited or were incorrect, such as words about clinical data, anamnesis and laboratory tests with molecular biology (polymerase chain reaction — PCR, the next big thing in medical practice).

Continuing with the theme of syphilis, we searched for the characteristics of the etiological agent Treponema pallidum. Again, some correct information, but as for current data on the possibility of a growing medium, nothing was mentioned. For some years Treponema pallidum has been cultivated. Research for vaccine development is being carried out(1,2).

In order to deepen our inquiry, on March 1st, 2023, we asked ChatGPT the following question: how can I diagnose the disease caused by Treponema pallidum in a laboratory?

The answer came in seconds.

Note that ChatGPT starts by saying that syphilis is a sexually transmitted disease and does not mention the possibilities of transmission by contaminated blood (transfusion, sharing of needles) or, mainly, by vertical transmission — from an infected pregnant woman to her fetus.

The system pointed to PCR for blood and CSF. When it cites clinical lesions, it mentions only primary syphilis, that is, the initial lesion of syphilis (hard chancre). However, in the outpatient practice of those who treat patients with syphilis with clinical manifestations, these are much more frequent in the secondary syphilis phase, that is, the recent phase of syphilis with pleomorphic lesions on the genitals, anus, the mouth of papular syphilids, flat syphilitic condyloma, among others.

ChatGPT did not say anything about the scraping of syphilis lesions for dark-field bacterioscopy, silver impregnation, and PCR in these materials.

It went on to omit data on a type of serology that is one of the most used today: chemiluminescence. This is an automated test that greatly speeds up serological screening — especially in laboratories with high demand.

Neither did it mention the quick test (treponemal) done with blood from a finger prick, especially for screening purposes, in the first consultation of pregnant women or in clinics for sexually transmitted diseases (STDs).

It also forgot to present elementary data on syphilis serology: the non-treponemal VDRL/RPR tests must mandatorily have their results, when reagents, also expressed in dilutions such as 1/1 (pure serum), 1/2, 1/4, 1/8…

With regard to data on syphilis gathered using artificial intelligence (AI), what we point out here are elementary questions for anyone who intends to offer something revolutionary in terms of the most up-to-date knowledge on the planet. At least for the year 2023.

So we insisted and asked ChatGPT another question.

When to use chemiluminescence serology and VDRL serology to screen and diagnose syphilis?

For ChatGPT to claim that treponemal serology for syphilis by chemiluminescence is aimed at monitoring response to treatment is an unforgivable mistake.

So, a doubt springs up in our minds: if AI, a state-of-the-art phenomenon, an activity that sucks data via robots on the internet, does not possess the latest information that has been published in scientific spheres, does not answer questions with traditional data as well as with the most current, what type of up-to-dateness and intelligence serves this product?

We believe that intelligent humans who are familiar with the topic will know right from wrong. However, those who know little or nothing about the researched subject can, with the “AI” of 2023, learn, write and publish serious untruths.

We believe (actually we are sure) that humans, the intelligent ones, will over time teach machines more and more how to respond better, with much more current data and correct information.

For now, all care is little. And we must be careful not to believe (and reproduce) information that in no way resembles correct data — in current jargon, fake news.

After presenting our concerns — those of graduated and specialized medical professionals from a time when studies were based on face-to-face lectures, with professors who wrote in chalk on the blackboard, using slides on a Kodak carousel projector and printed books — we searched the internet for recent publications on this new theme we dare to address.

ChatGPT is not the first, the only or the last “computer program that uses artificial intelligence to simulate a human conversation through text messages, voice messages or other means of communication”(3).

“ChatGPT (Chat Generative Pre-Trained Transformer) is a prototype of an artificial intelligence chatbot developed by OpenAI and specialized in dialogue. The chatbot is a language model tuned with supervised and reinforcement learning techniques. The base model that was tweaked was OpenAI’s GPT-3.5 language model, an improved version of GPT-3.

ChatGPT launched in November 2022 and drew attention for its detailed and articulate responses, although the accuracy of its information has been criticized.”(4).

The ChatGPT program is formed and trained based on large amounts of texts published online until 2021, such as news, books (millions), web pages, including Wikipedia and other sources of information(4).

In an excellent publication on Medscape, Gatusso says that “conversational agents can open up many perspectives in the field of health, providing assistance to patients in the first place, giving them information about diseases, treatments and medications. In the future, they should also play an important role in therapeutic education”(4).

For us, this can lead to a lot of confusion, as many lay patients (and even some health professionals) do not have the necessary training to reflect on medical information. It can also generate attitudes such as: “I will first check the chatbot/ChatGPT and do what is said there. If my condition doesn’t get better, I’ll see a doctor.” Many will also think about having the chatbot (still free) as the first step to avoid the waiting time of a doctor’s appointment (sometimes weeks), the time spent in the consultation, including commuting, waiting in the consulting room, and the money spent to pay for it.

On the other hand, as previously mentioned, today, in March 2023, ChatGPT is “fed” with information made available on the World Wide Web until 2021 (we do not know until which month). In this world where thousands of pieces of information are published every day, truths from 2021 may be false today and vice versa. We imagine that these programs capture more quickly those contents that are repeated more often. And it is now well known: fake news is repeated with frightening frequency.

In a relaxed interview, ChatGPT was extremely cautious: “It is unlikely that artificial intelligence will entirely replace doctors for diagnosis and therapeutic choice in the short or medium term”, it assured. “Health professionals have experience, and experience cannot be replaced by artificial intelligence. They are trained to interact with patients, understand the social and emotional factors that can influence people’s health and consider the particular characteristics of each patient”(4).

In yet another Medscape post, physician Alok Patel, reflecting on medical bornout, concluded that, “compared to a physician, chatbots are not capable of taking into account the individual characteristics of patients or the underlying context, considered in a detailed assessment. These tools are more ‘black and white’. They cannot be better than the information they have. There is also a concern about patient overtreatment if too many people start using these tools to treat symptoms. Computers are more accessible, and some patients may feel more comfortable typing their problems than talking to someone”(5).

Eliza was the first software for simulating dialogues, the so-called chatbots, the “conversation robots”. The name comes from the main character in the play Pygmalion, Eliza Doolittle, written by George Bernard Shaw in 1913(6).

The Eliza software was the first program for natural language processing in history and was created by Joseph Weizenbaum in the AI laboratory of the Massachusetts Institute of Technology (MIT), between the years 1964 and 1966. The basic idea was to simulate the conversation between man and machine, and the author himself was surprised by the number of people who attributed to Eliza characteristics very similar to human feelings(6).

Currently, on March 5th, 2023, several companies are proposing to launch competitors for ChatGPT, like Wenxin Yiyan in Chinese; or ERNIE Bot in English, scheduled for release in March 2023(3).

Today access is free. However, in the commercial world we live in, it is soon expected that access will be paid — especially for levels with more complex information and answers.

An AI law, which regulates the use of AI in healthcare, is due to be passed soon in Europe. Its implementation is scheduled for the end of this first semester(4).

We daily receive an email from The New York Times, and that of March 2nd, 2023, was signed by Lauren Jackson(7). She started with: “Good morning. Artificial intelligence is changing the way people work. This can be scary. But for now, most jobs are still safe.”

She went on to say, “AI already permeates our lives, powering voice assistants like Siri and Alexa, unlocking phones with facial recognition and autocomplete sentences in emails.”

About limitations, she was blunt: “As we’ve covered in this newsletter, AI chatbots are flawed. They tend to make mistakes — like the one that led to a $100 billion drop in Google’s cumulative stock market value when it appeared in an ad.”

She further argued that the chatbot is still the tool, not the creator(7).

It is important to make this clear, as, for many, this AI tool should not be seen as a final product, but rather as another source of inspiration to be analyzed and scanned for flaws.

Echoing the words of Eli Snyder, a special education teacher from Colorado (United States of America), Jackson(7) tells us: “AI cannot do Snyder’s job of teaching music and gym classes. You can’t play the piano or the HORSE basketball game, and you can’t facilitate students’ social and emotional learning. But it gives Snyder more time for that work.”(7)

“Everyone is talking about how AI is going to replace us,” Snyder said. “I do not agree. It will free up more time in our jobs to do other, more productive things.”(7)

To wrap up this editorial, we should mention the excellent material by Marlen Couto(8) published in O Globo: Nova Fronteira, “How ChatGPT can expand the scale of disinformation campaigns”(8).

In the journalistic article, the author describes several research studies that have used the aforementioned chatbot. She points out elementary errors and dubious positions in responses to typically Brazilian questions, such as: the suspicion about the electronic ballot boxes used in elections, the closing of the Federal Supreme Court, attacks on Minister Alexandre de Moraes, scientific denialism relating the COVID-19 vaccine to HIV, a text with conspiracy content that questions the roundness of the earth, among others. Such AI responses show that, on certain issues, ChatGPT can be a disinformation machine.

The world changes. People change. The rules change.

In the 1970s, medical courses and congresses were only allowed for medical students or trained physicians. In some of these events, students only entered if enrolled in the third (or more advanced) year of college.

The open internet and its range of scientific information (on Medicine and other sciences) offers a volume of data that can range from describing signs and symptoms to laboratory tests to be requested, therapeutic schemes, vaccination schemes and all the knowledge that a professional should have for good performance. These are things taught in classrooms, at medical events, in peer-reviewed scientific journals.

Of course, this AI network with millions (billions?) of pieces of information can be used judiciously, upon reflection, combined with an analysis of the clinical history of the patient, of his/her clinical examination, of detailed data on the reason for the consultation, of the history of the current illness, of his/her past pathological history, of the surgeries performed, of the history of drug hypersensitivity, of the sexual history, of illicit drug use, drug intolerance, and family history data… This type of information, in addition to a continuous exchange with people and their families, can undoubtedly add enormous value to the well-being of public health, of humanity.

However, if used robotically and by people without training in scientific, medical and psychological thinking, it can cause enormous damage to human relationships.

And things do not stop. There is already a chatbot (GPTZero) that detects material produced by AI. That is, a program can “find out” whether a given text was produced entirely or partially by AI. We tested GPTZero with a text produced by ChatGPT presented here. The answer was: “Your text is likely to have been written entirely by AI.” How much does this change (and influence) teachers, editors?

We would like to end by quoting a text written by the ChatGPT program itself: “It is unlikely that artificial intelligence will entirely replace doctors for diagnosis and therapeutic choice in the short or medium term. Healthcare professionals have experience, and experience cannot be replaced by artificial intelligence. They are trained to interact with patients, understand the social and emotional factors that can influence people’s health, and consider the particular characteristics of each patient.”(4)

As humans who claim to be intelligent beings, we must be permanently aware and in control, no matter how much money the humans in corporations want to make. Notice that ChatGPT quoted the words “entirely”, “short and medium term”. Does this robotic program “imagine” that in the long term doctors will be entirely replaced by machines?