Every day, people turn to generative artificial intelligence (AI) tools like ChatGPT, Gemini, and Claude for quick health answers, diet recommendations, and meal plans. To the average user, AI outputs can feel remarkable: responses arrive in seconds, structured neatly, and written with an authoritative, reassuring tone.However, beneath this polished exterior lies a significant health risk – AI-generated advice can sound confident even when it is inaccurate, incomplete, or potentially harmful.Large language models (LLMs) generate responses by learning statistical patterns from data. Although they can provide useful health information, a plausible answer is not necessarily an accurate or safe one. Models may overlook contraindications, misunderstand dietary restrictions, or present uncertain information without adequate caution.For health and nutrition professionals, understanding these limitations is increasingly important for supporting client safety and demonstrating why human professional judgement remains essential.<hr><h2 id="bc2f47a2-136e-49c0-88b5-67a9a25546e5" data-toc-id="bc2f47a2-136e-49c0-88b5-67a9a25546e5" style="text-align: left; font-size: 24px; line-height: 1.2;">Case Study 1: ChatGPT, Sodium Bromide, and Bromism</h2>One of the most striking examples of the risks of AI-generated health advice was documented in a 2025 medical case report.A man who wanted to eliminate chloride from his diet reportedly consulted ChatGPT about dietary alternatives. He subsequently replaced table salt (sodium chloride) with sodium bromide, a substance unsuitable for use as a dietary salt substitute.The consequences were serious.The case highlights three critical facts:<ol><li aria-level="1">Sodium bromide is not an appropriate replacement for dietary sodium chloride.</li><li aria-level="1">Excessive bromide exposure can accumulate in the body and cause toxicity.</li><li aria-level="1">Bromide intoxication, known as bromism, can cause severe neurological, psychiatric, and other systemic symptoms.</li></ol>Following the AI’s instructions, the man replaced his salt with sodium bromide. Weeks later, he was rushed to the hospital suffering from severe bromism (bromide intoxication).When treating physicians re-entered the exact same prompt into ChatGPT to investigate the origin of the advice, the chatbot repeated the identical dangerous recommendation – advising table salt substitution with sodium bromide without providing a single toxicity warning or medical disclaimer.The case illustrates an important danger: an AI-generated recommendation may appear scientifically plausible while overlooking essential safety information.For nutrition professionals, it also reinforces the importance of assessing the safety of any suggested dietary substitution rather than relying solely on the confidence or apparent technical accuracy of an AI response.Scientific reference: Eichenberger, A., Thielke, S., &amp; Van Buskirk, A. (2025). A Case of Bromism Influenced by Use of Artificial Intelligence. Annals of Internal Medicine: Clinical Cases, 4(8).&nbsp;<hr><h2 id="693e2ee7-5c26-4159-b84a-7c23dbf851fb" data-toc-id="693e2ee7-5c26-4159-b84a-7c23dbf851fb" style="text-align: left; font-size: 24px; line-height: 1.2;">Case Study 2: AI Meal Plans, Food Allergies, and Anaphylaxis Risks</h2>The safety limitations of general-purpose AI tools extend directly into dietary planning and allergen management.Research examining ChatGPT-generated dietary recommendations has identified examples of inappropriate foods appearing in meal plans for people with specific dietary restrictions.One documented example involved almond milk appearing in a nut-free meal plan.This is a potentially serious error. Almonds are classified as tree nuts for food-allergy purposes, and almond milk is unsuitable for someone with an almond allergy. For a susceptible individual, exposure could trigger a severe allergic reaction, including anaphylaxis.A meal plan can appear nutritionally balanced while still being unsafe for the person expected to follow it.This illustrates why an AI-generated meal plan must be checked against a client's allergies, medical history, dietary restrictions, and individual requirements before being considered for use.Scientific reference: Is ChatGPT an Effective Tool for Providing Dietary Advice? (2024). Available via PubMed Central: <a target="" href="https://pmc.ncbi.nlm.nih.gov/articles/PMC10892804/">https://pmc.ncbi.nlm.nih.gov/articles/PMC10892804/</a><hr><h2 id="2d81bea5-8186-4529-9b67-e22ee8fad434" data-toc-id="2d81bea5-8186-4529-9b67-e22ee8fad434" style="text-align: left; font-size: 24px; line-height: 1.2;">Why AI Can Give Unsafe Health Advice: Medical Benchmarks vs Real-World Practice</h2>Why can AI systems perform impressively on standardised medical examinations yet make serious mistakes in everyday health and nutrition scenarios?As Alex Ruani, health misinformation researcher at University College London and Chief Science Educator at The Health Sciences Academy, argues in an article published by the official BMJ Leader blog:&nbsp;<h3 id="9066f0b7-885b-4cfe-a9c7-3a2a4338487d" data-toc-id="9066f0b7-885b-4cfe-a9c7-3a2a4338487d" style="text-align: left; font-size: 20px; line-height: 1.2;">“ChatGPT is not your doctor, dietitian, or therapist.”</h3>The disconnect stems from how AI models are built, evaluated, and used.<h3 id="0bb3350c-5bcd-4e50-9179-71994c8be32e" data-toc-id="0bb3350c-5bcd-4e50-9179-71994c8be32e" style="text-align: left; font-size: 20px; line-height: 1.2;">1. Plausible answers are not the same as safe answers</h3>General-purpose AI is designed to generate useful, coherent responses. However, the ability to produce convincing language does not establish that an answer has undergone appropriate clinical safety assessment.A chatbot may answer the question it receives without identifying a dangerous assumption behind that question.For example, recommending a salt substitute requires more than recognising a chemical similarity. It also requires understanding human physiology, nutritional requirements, toxicity, and the individual's circumstances.<h3 id="53b14ee6-e008-4336-90a9-23473b6444e5" data-toc-id="53b14ee6-e008-4336-90a9-23473b6444e5" style="text-align: left; font-size: 20px; line-height: 1.2;">2. Passing medical exams is not clinical validation</h3>Claims that AI models have passed medical examinations or outperformed doctors on particular benchmarks can be misleading when presented without context.Multiple-choice testing on fixed question sets does not fully reflect the uncertainty and complexity of real-world healthcare.Actual clinical and nutrition practice involves medical histories, coexisting conditions, drug–nutrient interactions, individual variability, and decisions about when further assessment or referral is necessary.Medical examination performance should not be mistaken for evidence that an AI system can independently provide safe, personalised health advice.<h3 id="5dd7a383-4aaa-4c43-88cc-ad365572ac7a" data-toc-id="5dd7a383-4aaa-4c43-88cc-ad365572ac7a" style="text-align: left; font-size: 20px; line-height: 1.2;">3. Health AI requires more than general-purpose safety guardrails</h3>AI systems are not all developed, tested, or regulated to the same standards.Some healthcare-specific AI applications operate within defined clinical workflows and may be subject to additional evaluation, oversight, and regulatory requirements.General-purpose consumer chatbots, however, should not automatically be assumed to have undergone the same scrutiny as medical devices or professionally governed healthcare services.As Alex Ruani highlights, robust health-specific safety evaluation, transparent risk assessment, and appropriate governance are necessary as these technologies become more widely used.Further reading: <a target="" href="https://blogs.bmj.com/bmjleader/2025/09/19/chatgpt-is-not-your-doctor-dietitian-or-therapist-why-we-urgently-need-safety-evaluation-standards-for-generative-ai-in-health-but-who-will-take-the-lead/">Alex Ruani — Why We Urgently Need Safety Evaluation Standards for Generative AI in Health, BMJ Leader blog</a><hr><h2 id="cb04003f-0b63-4748-ad2b-1dd4d2c733f3" data-toc-id="cb04003f-0b63-4748-ad2b-1dd4d2c733f3" style="text-align: left; font-size: 24px; line-height: 1.2;">What AI Cannot Replace: The Human Advantage in Nutrition Practice</h2>While AI can quickly summarise general information, organise research, or assist with administrative tasks, these capabilities do not remove the need for qualified human judgement.Three professional competencies are especially important:<ul><li aria-level="1">Clinical risk awareness and appropriate referral: Recognising concerning symptoms, reviewing relevant health information, identifying potential risks, and knowing when a client needs assessment by an appropriately qualified healthcare professional </li><li aria-level="1">Personalised context: Understanding a client's lived experience, psychological relationship with food, socioeconomic circumstances, medical needs, and preferences. </li><li aria-level="1">Empathetic communication and behaviour change: Supporting clients through sustainable habit change using active listening, individual motivation, and non-judgemental guidance.</li></ul>AI can support parts of these processes, but it cannot independently assume the professional responsibility, accountability, or duty of care associated with regulated healthcare practice.As Maurice Castelijn, CEO of The Health Sciences Academy, emphasises:<h3 id="9e7d510a-cf90-4284-a1f9-05e0e87fddc4" data-toc-id="9e7d510a-cf90-4284-a1f9-05e0e87fddc4" style="text-align: left; font-size: 20px; line-height: 1.2;">“If it's not personalised, it's not effective. Clients don't just need fast answers – they need trusted human guidance, safety oversight, and accountability.”</h3>The practical distinction is not simply between humans and technology. It is between receiving an answer and being able to evaluate whether that answer is appropriate for a particular person.That is where evidence literacy, professional competence, and sound judgement become especially valuable.<hr><h2 id="bcaec822-2249-4741-81bb-dbd817788136" data-toc-id="bcaec822-2249-4741-81bb-dbd817788136" style="text-align: left; font-size: 24px; line-height: 1.2;">How Nutrition Professionals Can Respond When Clients Trust AI Advice</h2>When clients arrive with TikTok trends, Google search results, or ChatGPT meal plans, dismissing their sources outright may make productive conversations harder.A more constructive approach is to understand what the client has read, assess the underlying evidence, and explain any important limitations or safety considerations.For example, a nutrition professional can:<ol><li aria-level="1">Ask what the client was trying to find out. Understanding the original question helps reveal what information or reassurance the client needs.</li><li aria-level="1">Review the AI-generated recommendation together. Identify claims that are supported, uncertain, or inappropriate for the client's circumstances.</li><li aria-level="1">Explain the reasoning behind professional guidance. Help the client understand why individual context matters rather than simply telling them the AI is wrong.</li><li aria-level="1">Agree on an appropriate next step. Where a recommendation raises medical concerns, encourage assessment or referral within the relevant professional scope of practice.</li></ol>This approach helps clients develop confidence in evidence-based guidance without making them feel judged for using AI.It also gives practitioners an opportunity to demonstrate the value of scientific understanding, communication skills, and personalised support.<hr><h2 id="5288276a-42e8-40cd-a1f7-6f602b322f40" data-toc-id="5288276a-42e8-40cd-a1f7-6f602b322f40" style="text-align: left; font-size: 24px; line-height: 1.2;">Master the Art of Professional Authority in the Digital Age</h2>When clients enter your office armed with <a target="" href="https://webinars.thehealthsciencesacademy.org/cpd-ceu-what-to-do-when-clients-trust-tiktok-google-or-ai-more-than-you-and-how-to-become-the-expert-they-trust">TikTok trends, Google search results, or ChatGPT meal plans</a>, how do you respond without sounding defensive or alienating them?The Health Sciences Academy is hosting an interactive CPD/CEU webinar designed to help practitioners navigate client claims, communicate scientific evidence clearly, and build professional trust.<div style="display: flex;"><img src="https://ezycourse.b-cdn.net/5703/cmv7s75wm0e3g01zeh4xebwny.png" alt="image" style="position: relative; width: 662px; height: auto; cursor: pointer; margin: 0px auto;"></div><a target="" href="https://webinars.thehealthsciencesacademy.org/cpd-ceu-what-to-do-when-clients-trust-tiktok-google-or-ai-more-than-you-and-how-to-become-the-expert-they-trust">Reserve your free seat today</a>This article is intended for professional education and general information. It does not replace individual medical advice, diagnosis, or assessment by an appropriately qualified healthcare professional.<hr><h2 id="3045e7c8-6b06-4af1-aca1-da990c3975e9" data-toc-id="3045e7c8-6b06-4af1-aca1-da990c3975e9" style="text-align: left; font-size: 24px; line-height: 1.2;">Frequently Asked Questions About AI Health and Nutrition Advice</h2><h3 id="87af7165-f3e4-4b6f-8ec3-3a1b968d6afb" data-toc-id="87af7165-f3e4-4b6f-8ec3-3a1b968d6afb" style="text-align: left; font-size: 20px; line-height: 1.2;">Can AI tools like ChatGPT create safe, personalised diet plans?</h3>AI tools such as ChatGPT can generate meal plans and dietary suggestions, but their recommendations are not always accurate, safe, or appropriate for individual needs. They may overlook food allergies, medical conditions, medication interactions, or nutritional requirements. AI-generated meal plans should therefore be carefully reviewed, particularly when they involve medical dietary restrictions or complex health needs. Personalised nutrition guidance requires consideration of the individual's circumstances and, where appropriate, oversight from a qualified professional.’<h3 id="2e95050f-2374-413d-8c55-3be05156aa1b" data-toc-id="2e95050f-2374-413d-8c55-3be05156aa1b" style="text-align: left; font-size: 20px; line-height: 1.2;">Why does AI health advice sound so convincing even when it is incorrect?</h3>Generative AI models are trained to produce coherent, contextually appropriate language, but fluent explanations do not necessarily reflect accurate scientific reasoning. An AI system may present incomplete, misleading, or incorrect health information with the same confidence as a reliable answer. This can create a false impression of certainty, making it difficult for users to distinguish sound advice from potentially harmful recommendations.<h3 id="a9f1b896-8153-48d7-80f5-4e6f9e3df28b" data-toc-id="a9f1b896-8153-48d7-80f5-4e6f9e3df28b" style="text-align: left; font-size: 20px; line-height: 1.2;">Will AI replace qualified nutrition professionals and health coaches?</h3>AI is changing how people access nutrition information and how practitioners work, but generating dietary advice is only one part of professional practice. Qualified practitioners bring evidence appraisal, individual assessment, appropriate risk awareness, empathetic communication, and behaviour-change skills. AI can support some of these activities, but it cannot independently assume professional accountability or responsibility for client care. Practitioners who understand both nutrition science and the limitations of AI will be better placed to use these tools responsibly.<h3 id="c83e45f9-5251-4769-b282-6249d5e29d0e" data-toc-id="c83e45f9-5251-4769-b282-6249d5e29d0e" style="text-align: left; font-size: 20px; line-height: 1.2;">How should I respond when a client brings a ChatGPT diet plan into a consultation?</h3>Start by asking what the client wanted to learn and what appealed to them about the AI-generated advice. Rather than dismissing the plan, review its recommendations together, assess the supporting evidence, and explain any safety concerns or important individual considerations. This creates an opportunity to demonstrate the value of personalised guidance while maintaining the client's trust and involvement in decisions about their health.<h3 id="d73c575a-6907-47f0-b790-675197b823a8" data-toc-id="d73c575a-6907-47f0-b790-675197b823a8" style="text-align: left; font-size: 20px; line-height: 1.2;">Is it safe to use AI in my own nutrition practice?</h3>AI can support nutrition professionals with tasks such as drafting educational materials, organising information, and preparing communication resources. However, its outputs require appropriate professional review, especially when they involve health-related recommendations. Practitioners should not rely on general-purpose AI as an independent authority for diagnosis, interpretation of clinical findings, or decisions outside their professional scope. Confidential client information also requires careful handling in accordance with applicable privacy requirements and organisational policies.

Every day, people turn to generative artificial intelligence (AI) tools like ChatGPT, Gemini, and Claude for quick health answers, diet recommendations, and meal plans.

The Illusion of Accuracy: Why Generative AI Gives Dangerous Health Advice with Confident Style

Case Study 1: ChatGPT, Sodium Bromide, and Bromism

Case Study 2: AI Meal Plans, Food Allergies, and Anaphylaxis Risks

Why AI Can Give Unsafe Health Advice: Medical Benchmarks vs Real-World Practice

“ChatGPT is not your doctor, dietitian, or therapist.”

1. Plausible answers are not the same as safe answers

2. Passing medical exams is not clinical validation

3. Health AI requires more than general-purpose safety guardrails

What AI Cannot Replace: The Human Advantage in Nutrition Practice

“If it's not personalised, it's not effective. Clients don't just need fast answers – they need trusted human guidance, safety oversight, and accountability.”

How Nutrition Professionals Can Respond When Clients Trust AI Advice

Master the Art of Professional Authority in the Digital Age

Frequently Asked Questions About AI Health and Nutrition Advice

Can AI tools like ChatGPT create safe, personalised diet plans?

Why does AI health advice sound so convincing even when it is incorrect?

Will AI replace qualified nutrition professionals and health coaches?

How should I respond when a client brings a ChatGPT diet plan into a consultation?

Is it safe to use AI in my own nutrition practice?

Explore science-based articles on nutrition, health, wellness, behaviour change, clinical practice, dietary supplements, gut health, sports nutrition, menopause, and professional development.

Nutrition Blogs | Evidence-Based Health and Nutrition Articles | THSA

Discover science-based nutrition articles, research insights, clinical practice tips, and professional development resources from THSA.

The Illusion of Accuracy: Why Generative AI Gives Dangerous Health Advice with Confident Style

The Illusion of Accuracy: Why Generative AI Gives Dangerous Health Advice with Confident Style

Related Posts

Why Perfectionism Can Keep Nutrition Professionals Stuck in Learning Mode

Good Nutrition Advice Isn’t Always Enough: 5 Client Scenarios That Show What Can Get in the Way of Lasting Change

Which Nutrition Certification Is Right for You? A Practitioner’s Guide to Specialisation and Revenue Growth

Why Good Nutrition Advice Doesn’t Always Lead to Lasting Change – and How to Build a Client Journey That Works

Coronavirus: Misinformation and setting the record straight