A hiring manager at a large tech company told me they had four hundred engineers working on their AI platform and zero people with training in philosophy, ethics, or the social sciences. When I asked who was responsible for deciding what the model should optimize for, he looked confused. The engineers were responsible. They were optimizing for accuracy.

This is the structural gap in most AI teams. The people building the systems are trained to answer “how” questions — how to improve accuracy, how to reduce latency, how to scale inference. The questions that matter most — what should the system optimize for, whose values does it encode, what happens when optimization conflicts with fairness — are “should” questions. Engineering training does not prepare people to answer them.

The optimization target problem

Every AI system optimizes for something. A recommendation engine optimizes for engagement. A content moderation system optimizes for a definition of safety. A hiring model optimizes for a definition of candidate quality. A fraud detection system optimizes for a balance between catching fraud and not inconveniencing legitimate customers.

Choosing the optimization target is not an engineering decision. It is a value judgment. Engineers are well-equipped to build systems that achieve a given optimization target efficiently. They are not trained to evaluate whether the target is the right one, what the secondary effects of optimizing for it will be, or whose interests are served by the choice.

Consider engagement optimization. Engineers can build recommendation systems that maximize time-on-platform with remarkable effectiveness. Whether maximizing time-on-platform is good for users, for society, or even for the platform’s long-term health is a question that requires a different kind of training. It requires the ability to reason about values, tradeoffs, and the difference between what people click on and what is good for them.

Philosophers spend their careers on exactly these questions. The entire field of ethics is dedicated to reasoning about what outcomes we should pursue, how to weigh competing values, and how to think clearly about tradeoffs that have no clean resolution. This is not a soft skill. It is the hardest skill in AI system design, because getting it wrong produces systems that work exactly as specified and cause harm anyway.

The epistemology gap

AI systems make claims about the world. A medical model claims that a patient has a particular risk level. A content model claims that a piece of text is or is not misinformation. A hiring model claims that a candidate is or is not qualified. Each of these claims carries an implicit theory of knowledge: what counts as evidence, how much uncertainty is acceptable, and what the relationship is between statistical patterns and truth.

Epistemology — the branch of philosophy concerned with knowledge, belief, and justification — provides the tools for reasoning about these questions rigorously. Engineers are trained in statistics, which addresses some of these concerns. But statistics is a tool, not a framework for reasoning about what it means for a model to “know” something, or whether a statistical pattern constitutes sufficient justification for a consequential decision.

I have seen the consequences of the epistemology gap in practice. A healthcare company built a risk prediction model with 89% accuracy on a held-out test set. The engineering team reported the model as “accurate” and pushed for deployment. A clinician on the review panel asked a different question: “When this model is wrong, what kind of wrong is it, and what happens to the patient?” The model’s errors were concentrated in a patient population with a specific comorbidity pattern. Deploying the model without addressing this error pattern would have systematically under-assessed risk for the patients who were most vulnerable.

The engineering team’s training equipped them to measure accuracy. It did not equip them to ask whether accuracy, as measured, was the right epistemic standard for the deployment context. That question required a different kind of thinking.

What philosophers actually do on AI teams

The suggestion to hire philosophers on AI teams usually gets one of two reactions: skepticism (what would they do all day?) or caricature (they would slow everything down with abstract debates). Neither reaction is accurate in practice.

Philosophers on AI teams serve three functions that engineers are not trained for.

They interrogate the optimization target. Before an engineering sprint begins, someone should be asking: what are we optimizing for, why, who benefits, who is harmed, and what happens when the optimization conflicts with other values we hold? This is not a one-time activity. It is an ongoing practice, because the implications of an optimization target become clearer as the system interacts with the real world.

They reason about edge cases that are value-laden, not just technical. Engineers think about edge cases in terms of input distributions — what happens when the input is null, out of range, or adversarial. Philosophers think about edge cases in terms of values — what happens when the model’s recommendation conflicts with a user’s autonomy, when the model’s definition of “fair” conflicts with a community’s definition, when the model’s accuracy is high overall but low for a specific population. These are the edge cases that cause public backlash, regulatory scrutiny, and genuine harm.

They translate between technical and ethical reasoning. Most AI ethics discussions fail because they happen in either purely technical or purely ethical language. Engineers discuss fairness metrics. Ethicists discuss justice. Neither group can connect their framework to the other’s. People trained in both traditions bridge this gap, making the ethical concerns concrete enough for engineering decisions and the engineering constraints visible enough for ethical reasoning.

The counterargument and its limits

The strongest counterargument is practical: AI teams are already expensive, and adding non-engineering roles increases cost without increasing output. This argument assumes that output is measured in model accuracy or feature velocity. If output is measured in the quality of the systems that reach production — including their safety, fairness, and alignment with human values — then the investment in non-engineering roles is not a cost increase. It is a risk reduction.

The second counterargument is that engineers can learn the relevant ethical reasoning without hiring specialists. This is partially true. Some engineers develop strong ethical reasoning skills through self-study or professional development. But asking engineers to acquire expertise in philosophy, on top of their engineering expertise, is no more reasonable than asking philosophers to acquire expertise in distributed systems. Specialization exists for a reason.

The hiring signal

When I evaluate whether an AI team is taking the “should” questions seriously, I look at the team composition. If the team is entirely engineers and product managers, the optimization target was almost certainly chosen for technical convenience or business metrics without rigorous interrogation of its broader implications. If the team includes people trained in ethics, social science, or philosophy — not as consultants or reviewers, but as full team members with decision-making authority — there is a reasonable chance that the system’s values were considered alongside its accuracy.

The provocation: if your AI team has ten engineers and no one who has read Rawls, Sen, or Nussbaum, you are building a system that optimizes for a target that was never rigorously examined. The target might be fine. But you do not know that, and your engineering training does not equip you to evaluate it.