The following article was written by Dr. Cornelia C. Walther, a visiting scholar at Wharton and director of global alliance POZE. A humanitarian practitioner who spent over 20 years at the United Nations, Walther’s current research focuses on leveraging AI for social good.
Every boardroom has heard it. Every procurement checklist includes it. Every investor deck now features it somewhere between the mission statement and the revenue model. “AI ethics” has become the phrase that signals seriousness without necessarily delivering it. That gap deserves examination — because the weakness of the phrase is structural, and structural weaknesses tend to surface at expensive moments.
The two words — “intelligence” and “ethics” — contain separate category errors. Fix those errors, and a more useful question emerges: one that belongs at the beginning of AI strategy, before the build, before the contract, and well before the incident.
The First Misnomer: Intelligence
Call it a useful fiction that escaped its cage. “Artificial intelligence” began as a research shorthand and became infrastructure. Along the way, the word “intelligence” acquired a passenger it never asked to carry: the assumption that what machines do resembles what humans do, only faster, better, more powerful.
t does not. Current AI systems predict, classify, generate, and optimize. They perform specific tasks at remarkable speed and stumble on questions that require context, embodiment, or lived meaning. The 2026 Stanford AI Index documents organizational AI adoption at 88%, with four in five university students using generative AI. That is the footprint of a technology that has moved from tool to cognitive environment. In this context it is important to keep in mind the fundamental differences of natural and artificial intelligence.
Natural intelligence is a living process, shaped by aspiration, emotion, thought, and bodily sensation. It develops through action and consequence, through social belonging and moral formation, through the long curriculum of getting things wrong. Hybrid intelligence distinguishes carefully between natural intelligence — rooted in human biology, biography, and community — and the computational simulation of selected outputs of that intelligence. The distinction matters because organizations that confuse the simulation with the thing itself tend to misallocate authority. They let systems carry weight that requires human judgment, then express surprise when the outputs are technically correct and humanly wrong. The word “intelligence” in AI flattens what the natural cognitive eco-environment encompasses.
An automated hiring tool that filters variables for gender or geography by proxy is not exhibiting bad ethics. It is executing good math on bad assumptions. The ethics problem sits upstream, in the human decision to automate that particular judgment, on that particular dataset, toward that particular organizational goal. The 2021 paper “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” named this plainly: Fluent language is not understanding, and treating it as such generates costs that land — unevenly — on real people.
It is important to keep in mind the fundamental differences of natural and artificial intelligence.
The Second Misnomer: Ethics
The second compression happens to the word “ethics.” In corporate life, ethics has become compliance: a set of rules, reviews, and risk categories designed to keep organizations out of trouble. That is a legitimate function. It is also a significant narrowing of what ethics actually asks.
Doing right by people predates machine learning by several millennia. What AI introduces is a new operating environment for an old question: What do we owe one another? Which decisions deserve human deliberation, and which can be delegated to pattern-matching at scale? Ethics is not a new problem that emerged when large language models arrived. It is the oldest problem of humanity, now running inside recommendation engines, clinical triage systems, automated procurement, personalized learning platforms, and intimate consumer chatbots.
The checklist approach — fairness metrics, bias audits, explainability scores, accountability trails — offers genuine value. These instruments form part of the necessary infrastructure. They become thin when organizations treat them as the destination. The harder work involves asking what kind of intelligence a system cultivates in the people who use it daily. Does a workplace AI tool sharpen judgment or gradually outsource it? Does a customer-facing system build trust or undermine it? Does an educational platform deepen curiosity or manufacture the performance of engagement? Every system teaches something through repeated contact. Most organizations are not measuring the influence that their systems have on their people.
This is the zone that governance frameworks approach but rarely enter. UNESCO’s Recommendation on the Ethics of Artificial Intelligence places human dignity and oversight at the center. The OECD AI Principles call for trustworthy AI that supports well-being and sustainable development. The EU AI Act frames the European approach around human-centric design. These are necessary architectures. The cultural and practical question they leave open is this: What kind of human being — and what kind of organization — does sustained use of a given AI system produce?
What AI introduces is a new operating environment for an old question: What do we owe one another?
The Business Case for Getting This Right
If we reframe the misnomers, an intriguing business logic appears. An organization that treats AI as a simulation of selected human capacities, rather than a substitute for them, designs its deployment differently. It keeps humans in the loop on decisions where context, dignity, and consequence matter. It measures the impact of AI use on employee agency, not just employee efficiency. It asks whether AI tools are narrowing or expanding the range of thinking that happens in the organization. Staff well-being and performance improve in sustainable ways.
An organization that treats ethics as upstream architecture, rather than downstream audit, builds AI strategy around purpose from the start. The Prosocial AI Index offers a disciplined framework for this: a structured assessment asking whether AI systems are tailored, trained, tested, and targeted to bring out the best in — and for — the people and the planet. It turns moral aspiration into a practice of monitoring, learning, and correction, across the dimensions of purpose, people, profit, and planetary impact.
The business risks of ignoring this accumulate slowly. Agency decay — the gradual erosion of human judgment capacity through chronic delegation to machines — does not appear on a quarterly dashboard. Neither does the narrowing of organizational imagination, the hollowing of professional expertise, or the slow substitution of data confidence for actual wisdom. These are long-horizon vulnerabilities. Leaders who think in decades, not quarters, tend to find them interesting.
The Question That Matters
“AI ethics” will remain the phrase in circulation. The work it points toward is larger than the phrase suggests. The question for every executive, board member, and strategy team is specific: What kind of natural intelligence (NI) is being cultivated by our AI systems? This question belongs at the beginning of any AI strategy. It belongs in procurement criteria, in design briefs, in performance frameworks, and in the conversations that happen before any contract is signed. The answer shapes how we look at two subsequent interrogations: Are our people becoming more capable, more discerning, and more accountable with these tools — or less? Who are we becoming, as an organization, through sustained contact with the systems we are building and buying?
Four Questions to Rescope Your Relationship With Your NI and Your AI:
- Why are you using AI, individually and as a team? Go beyond the general efficiency and effectiveness arguments to look at the actual reasons.
- Who are you without your tools? What makes you unique as a person, as a team leader?
- Where do you stand on your AI journey? Have you moved to a stage where your assets are shaping your thinking?
- What are you doing to align your aspirations with your algorithms, and to avoid the reverse?
Source: View the original article at Knowledge@Wharton.
