A study has examined the reliability of large language models as medical assistants for the general public. The results suggest that while these models may perform well in tests, they struggle with real-world medical decisions and can be prone to errors, potentially due to user misinterpretation or limitations in handling complex medical questions. The findings indicate that relying solely on AI chatbots for medical advice may not be advisable.