BLOG# 5: Do current moral theories cover emotional AI, or do we need new ethics for non-human intelligence?

The way people make chatbots for cancer care shows we need AI that really thinks about what people need. These chatbots should be kind, give right information, and help each patient different. Cancer patients and their families are very scared and stress, they need someone to answer their questions. So chatbots can help them a lot. But now big AI models like GPT-3 and GPT-4 are coming into healthcare, and that bring problems. These chatbots must follow rules like being fair, honest, and keep patient info private. They should not treat some people bad or leave out poor communities.

This study looks at how AI like big language models work in cancer chatbots. The goal is to see the right and wrong of using AI in such sensitive place like cancer treatment. The study also looks at bias in the data that train AI , if the data is no good, the AI give wrong answers. The paper find big ethical problems and say we should use more different kinds of data, check the AI all the time, and make training better. Because AI is used more and more, we need to make sure it is safe and clear. The rules for making AI should come from good ethics and respect for human value.

Artificial intelligence (AI) hallucinations, where models produce plausible but incorrect or fabricated outputs, pose significant challenges for high-stakes applications such as healthcare, scientific writing, and legal decision-making. These errors arise from limitations in large language model (LLM) training data, which may contain inaccuracies, biases, or irrelevant information. Hallucinations can occur in closed-domain settings, where errors are easier to detect, or in open-domain contexts, which are more complex and harder to verify. Mitigating these risks requires robust training on accurate data, fact-checking mechanisms, transparency, and continuous monitoring of AI outputs. Collaborative efforts among developers, users, and regulators, coupled with education on AI limitations, are essential to ensure trustworthy, reliable, and ethically aligned AI systems across critical domains.

The rise of AI ethics forces us to confront philosophical tensions, particularly the boundary between human and non-human. As AI systems increasingly replicate human abilities such as reasoning, language, and creativity, traditional markers of humanness are challenged, revealing the need for a broader, pluralistic conception of human worth that recognizes diverse abilities and identities. Autonomy is another key concern, as highly autonomous AI in domains like healthcare and defense raises ethical questions about the limits of machine decision-making and the necessity of human-centered safeguards. However, human-centeredness is not neutral—it can reflect societal biases, privileging certain groups or species while reproducing discrimination against marginalized people or non-human animals. Anthropocentric assumptions often underpin AI design, risking unethical outcomes if left unexamined. This paper proposes a nuanced approach, viewing human-centeredness as a spectrum rather than a binary, allowing for the integration of human values while mitigating harms of anthropocentrism. By distinguishing human-centeredness from anthropocentrism, the paper emphasizes ethical AI that considers the moral standing of diverse human groups, marginalized populations, non-human animals, and the broader environment, ultimately promoting fairness, inclusivity, and responsible AI deployment.

The content of ethical standards is often interpreted as exclusively a matter of fairness, which is primarily taken to be a relational concern with how some people are treated compared with others. Illustrations of AI-based technology that raise fairness concerns include facial recognition technology that systematically disadvantages darker-skinned people or automated resume screening tools that are biased against women because the respective algorithms were trained on data sets that are demographically unrepresentative or that reflect historically sexist hiring practices. “Algorithmic unfairness” is a vitally important matter, especially when it exacerbates the condition of members of already unjustly disadvantaged groups. But this should not obscure the fact that ethics also encompasses nonrelational concerns such as whether, for example, facial recognition technology should be deployed at all in light of privacy rights or whether it is disrespectful to job applicants in general to rank their resumes by means of an automated process.

lastly, the emergence of emotionally responsive artificial intelligence represents one of the most complex and thought-provoking developments in modern technology. As explored across these blogs, AI is no longer limited to performing tasks or solving problems—it is now entering the deeply human space of emotion, empathy, and moral interaction. As a student reflecting on this, I find this both exciting and deeply concerning. It shows how far innovation has come, but it also reveals how unprepared we may be for its ethical consequences.

One of the key ideas that stands out is that emotionally responsive AI can imitate human feelings, but it cannot truly experience them. It can say “I understand,” recognize sadness in a voice, or respond in a comforting way, but all of this is based on patterns, data, and programming—not genuine emotional awareness. This creates what can be described as an “illusion of empathy,” where users may feel emotionally supported even though there is no real understanding behind the response. While this illusion can sometimes be helpful, especially in areas like mental health support or education, it also raises serious concerns about authenticity and trust. If people begin to rely heavily on AI for emotional connection, they may slowly lose the value of real human relationships.

Another important issue is moral responsibility. Even if AI systems appear to make decisions based on emotions, they are not moral agents in the true sense. They do not have intentions, conscience, or accountability. Therefore, responsibility cannot be placed on the machine itself. Instead, it lies with the humans who design, train, and deploy these systems. This includes developers, organizations, and even policymakers who regulate their use. As AI becomes more integrated into sensitive areas like healthcare and counseling, ensuring accountability becomes even more critical.

My blog highlight how emotionally responsive AI challenges traditional human-centered ethical theories. These theories are built on human experiences such as empathy, intention, and moral judgment. However, AI does not fit neatly into these categories. This creates a gap where existing ethical frameworks may not fully address the realities of human-AI interaction. As a result, there may be a need to expand or adapt these frameworks to include non-human systems that can influence emotions and decisions without actually possessing moral understanding.

At the same time, it is important to recognize that AI is not entirely negative. When designed responsibly, it can enhance human life, improve access to services, and support emotional well-being. The key issue is balance. AI should act as a tool that supports human connection, not as a replacement for it. Transparency, ethical design, and user awareness are essential in maintaining this balance.

Ultimately, this topic shows that the future of AI is not just about intelligence, but about humanity. As we continue to develop machines that can “act” empathetic, we must not forget what real empathy means. As a student, I believe the responsibility lies with our generation to ensure that technology grows alongside ethical awareness. If we fail to do this, we risk creating a world where emotions are simulated, relationships are artificial, and genuine human connection slowly fades away.

SOURCES

1.https://link.springer.com/article/10.1007/s11245-025-10324-y

2.https://www.amacad.org/publication/daedalus/artificial-intelligence-humanistic-ethics

3.https://link.springer.com/article/10.1007/s11245-025-10324-y

4.https://www.unesco.org/en/artificial-intelligence/recommendation-ethics