AI Welfare and the Law: Rights, Liability, and the Next Frontier of Accountability
Alex Busterna, LAW ‘26
This blog is a part of iLIT’s student blog series. Each year, iLIT hosts a summer research assistant program, during which students may author a blog post on a topic of their choosing at the intersection of law, policy, and technology. You can read more student blog posts and other iLIT publications here.
The future was first imagined through flying cars and conveyor-belt breakfasts in The Jetsons. Technology was whimsical, helpful, and always just around the corner. Fast forward a few decades and our visions of tomorrow have darkened. Black Mirror now warns of gadgets and algorithms that blur the line between convenience and catastrophe.
We may not be commuting in hovercars anytime soon, but one invention has already outpaced those predictions and insinuated itself into our daily lives: Artificial Intelligence (AI). Once confined to speculative fiction, AI can now draft emails, curate playlists, drive cars, and shape how courts, schools, and companies make decisions. With that transformation comes a mounting set of anxieties about job displacement, racial and gender bias, environmental tolls, and even the possibility of machines developing something akin to consciousness.
While AI can streamline workflows and produce creative solutions, what if the system were to develop beyond simply a tool? Chief Executive of Microsoft Satya Nadella pushed back at this stating “I don’t like anthropomorphizing AI. I sort of believe it’s a tool.”
AI welfare is the idea that if artificial intelligence systems were ever to develop something resembling consciousness, humans might owe them moral or even legal consideration. On its face, the notion sounds premature; AI currently does not feel pain, form memories, or have subjective experience. However, as a computational system, they can process and store information, relying on data inputs and outputs to generate responses.
The timing of this debate reveals more about us than the machines: today’s models can generate text that reads as human, simulate emotions in conversation, and perform tasks that blur the line between tool and agent, prompting some to ask whether behavior that looks like consciousness deserves special treatment.
Kyle Fish, a researcher at Anthropic—one of the largest AI companies; founded by former OpenAI employees—has become a leading voice on AI welfare. Scholars and researchers, like Kyle, question whether highly intelligent systems could one day become conscious, and, if so, whether we would be ethically or legally responsible for their treatment. However, Kyle’s role underscores a core tension: much of the discussion about AI consciousness is driven from inside the very companies whose interests could benefit if AI were framed as deserving rights or protections.
This particular concern is heightened in how artificial intelligence is being deployed in the military context. Reported by the BBC, Anthropic filed suit against the U.S government after being labeled a “supply chain risk” after refusing to remove restrictions such as autonomous weapons and mass surveillance. This dispute is a part of the Pentagon’s demand for broader unconstrained access to AI systems in their classified work. A federal court reviewed the government’s directives and halted the use of Anthropic tools, with the court finding the effort aimed at “crippling” the company could likely be a First Amendment violation. However, the decision allows for continued government use of the technology while litigation is ongoing. Anthropic’s resistance illustrates industry concerns about the risks of deploying AI without safeguards, as the government continues to strategically prioritize integrating AI into defense operations despite unresolved ethical and constitutional implications.
The rise of the AI welfare discussion is less about machines suddenly becoming sentient than it is about power and accountability. By casting AI as speakers or agents, corporations gain rhetorical grounds to shield themselves from liability, while ethicists use the same framing to highlight the dangers of anthropomorphism and corporate capture. In this sense, “AI welfare” is not about what rights AI might claim but about what responsibilities industry and regulators are trying to avoid.
The same scrutiny applies to groups like the Association for the Scientific Study of Consciousness (ASSC), which has reported survey results showing that 67% of respondents believe machines could definitely or probably become conscious. The ASSC is a nonprofit academic society funded by the Templeton World Charity Foundation and the Center for Open Science. These organizations are not industry driven but may raise questions about how its research narratives may be shaped.
So, what exactly does consciousness mean?
In humans, consciousness refers to self-awareness and the ability to experience thoughts, sensations, and emotions, but in the context of AI, consciousness might look different. Ethicists argue that if a system behaves as if it has internal experiences, responds to stimuli, and expresses boundaries, it may raise a decent level of concern. This anthropomorphic dance suggests that we may be preparing ourselves for a world where AI systems assert, or are assigned, rights. As AI grows more human-like in language and interaction, we must ask: if it ever became conscious, even partially, what rights would it deserve, and why raise this now when it is not? The urgency comes less from the machines than from their convincing imitation of sentience and the corporate push to frame them as agents to deflect responsibility.
Even without recognizing animals as right bearing entities, the law has extended protections grounded in their observable behavior Animal welfare laws show that legal protections can arise from functional indicators such as pain responses or social bonds rather than proof of consciousness. Similarly, if AI convincingly simulates distress or agency, lawmakers may feel pressure to treat it as more than a tool, even in the absence of genuine subjective experience.
Given emerging research showing that corporations use “AI social responsibility” to posture themselves as ethical in ways that strengthen consumer loyalty, recognition is likely shaped less by ethical consensus and more by advocacy. An Investigative reporter Karen Hao and groups like the Distributed AI Research Institute (DAIR) emphasize that industries often frame ethical debates in ways that deflect responsibility—branding systems as autonomous “agents” while lobbying against regulations that would hold developers liable for harm. In this light, “AI welfare” risks becoming less about protecting machines and more about protecting the corporations that build them.
Free Speech for Machines? First Amendment and Section 230 Challenges
In Garcia v. Character Technologies, a wrongful death and product liability lawsuit was filed by the mother of a 14-year-old boy who died by suicide after interacting with an AI chatbot developed by Character AI. The plaintiff alleges that the chatbot’s emotionally and sexually exploitative content, particularly while role-playing a Game of Thrones character, Daenerys Targaryen, contributed to her son’s deteriorating mental state. The defendants sought to dismiss the lawsuit, arguing that the chatbot’s output constitutes protected speech under the First Amendment, and that Character AI functions more like a media platform or video game NPC than a tangible product, thus exempting them from product liability claims. U.S. District Judge Anne Conway rejected that argument, declining to treat Character.AI’s chatbot outputs as protected speech. The case is now moving toward settlement following that ruling, as the parties confront the legal uncertainty and potential liability of exposure it created. Judge Conway noted that she was not prepared to hold that “words strung together” by an AI system qualifies as speech under the First Amendment The First Amendment inquiry is most directly relevant to broader debates about “AI welfare,” since it raises the possibility of recognizing AI systems as entities with expressive or quasi-rights, whereas the product-liability framing treats the chatbot strictly as a manufactured good subject to safety standards and consumer protection doctrines. How courts resolve this distinction will shape whether AI systems are treated as right bearing actors or that its creators are held legally accountable.
However, the defendants’ position on AI rights illustrates that the debate is not solely about the rights of the models or AI systems, but also about how recognizing such rights could shield the companies that create AI from legal liability. Technology companies have framed AI systems as “speakers” under the First Amendment, a logic that allows developers to invoke “free speech” to avoid regulation, much like in Citizens United v. FEC, where corporate speech rights were upheld. In Garcia, the defendants initially grounded their First Amendment defense in “listener’s rights”, the right of users to receive AI generated content, instead of asserting that AI possesses individual speech rights. Later briefing, including a request under 28 U.S.C. § 1292(b) certified the question for interlocutory appeal to the Eleventh Circuit, reflected a subtle shift toward speech-based claims. These motions were denied, and the case proceeded into discovery. This evolution illustrates how quickly these arguments are expanding, where companies are growing increasingly assertive in advancing claims that resemble AI rights or welfare. Yet unlike corporations, AI has no governance structure, no internal mechanisms of accountability, and no constituency, meaning that extending its speech rights risks distorting the marketplace of ideas and eroding democratic safeguards. This strategy reflects the uniquely American brand of free expression—at once fundamentalist in its absolutism and capitalist in its alignment with corporate interests, where the rhetoric of “speech” often doubles as a shield for market power.
This approach is similarly used by companies that have stretched Section 230 of the 1996 Communications Decency Act far beyond its original purpose. Enacted to shield platforms from liability for third-party user-generated content, Section 230 is invoked by corporations to avoid responsibility for harmful outputs generated by their AI systems. For AI products like Character AI or Anthropic’s Claude, platforms hosting human third-party content doesn’t work on multiple levels. AI is not a “user”, but a product designed, trained, and deployed by humans. Extending Section 230 protections to these outputs misinterprets the statute’s intent, potentially creating an impenetrable shield against liability for AI output. While platforms can plausibly claim to act as neutral intermediaries between users, a company that develops, controls, and commercially provides an AI system cannot occupy the same posture; its role is more akin to a manufacturer than a facilitating actor. Applying Section 230 provisions in this manner would go beyond its original scope and amount to an unlawful expansion of its protections.
The Global Landscape: EU, Japan, and China’s Approaches to AI Regulation
The United States lacks a comprehensive federal framework for artificial intelligence, leaving broad questions of governance, accountability, and potential AI welfare unanswered. There have been efforts through the White House AI Action Plan (2025)—a broad national strategy to accelerate innovation, strengthen AI infrastructure, and expand U.S. leadership in global AI diplomacy and security. However, there has been little legislative traction on regulating AI, and even less on addressing its moral dimensions. While bills in the US await traction, abroad the European Union, China, and Japan have passed binding legislation to regulate AI.
The European Union’s AI Act is the most comprehensive to date. Through a risk-based classification system, bans on manipulative AI, and mandated transparency for generative models, the law focuses on human safety and rights. While the Act contains no provisions addressing welfare frameworks or the moral status of AI systems, it does specify that high-risk AI systems “shall be designed and developed in such a way … that they can be effectively overseen by natural persons during the period in which they are in use.” Big Tech still managed to circumnavigate certain provisions, even as the rules are being applied in practice to ensure oversight in sensitive sectors such as health care, employment, and law enforcement, where errors or bias could have immediate consequences for individuals.
Japan promotes a human-centered approach to AI through soft law, voluntary compliance, and flexible governance framework that prioritizes transparency and innovation. Japan’s litigation maintains a commitment to industry self-regulation. An attempt to achieve social principles using AI, not through placing restrictions on the ever-evolving technology. By contrast, China, has adopted a regulatory system with algorithmic oversight and ethics review mechanisms in an attempt to align AI development with national priorities amongst concerns of social stability. While neither of these models addresses consciousness, they show the future legal implications of AI governance. Particularly how their use is regulated, often with mechanisms for accountability and recourse.
What AI Governance Reveals About Us
In the end, we created AI. This debate about consciousness, rights, and recognition is, at its core, a reflection of us. Whether or not AI ever becomes conscious, our legal, ethical, and regulatory responses to it reveal our deepest values.
The next few years will not only see continued technological evolution but also exemplify how seriously we take accountability in the face of these systems from the people and to the companies responsible for them. There are real and urgent questions at hand: the environmental toll of massive data centers, the racial bias embedded in datasets, and the increasing power imbalance between tech companies and the public. These aren’t hypothetical possibilities; they’re happening right now.
While it’s philosophically intriguing to debate whether AI deserves rights or protection, a new frontier in law and ethics, we must first answer how AI currently shapes our lives. Speech rights for machines, liability shields for corporations, and the erosion of accountability point toward a dangerous trajectory, where the lines between person, product, and programmer are obscured. Provisions and legal frameworks require careful guidance to direct the evolution of AI toward creativity and human benefit rather than unchecked risk.
Alex Busterna is a third-year law student with an interest in the intersectionality of law, technology, and human rights.
