The limits of AI and ML sort into five checkable categories: data that decides what a model can know, engineering walls like overfitting and compute cost, trust failures in the black box and in ourselves, human limits the law now guards, and frontier failures like hallucination, with the speculative end labeled separately.
- Bias is learned from data: Amazon’s 2015 resume screener penalized the word "women’s", and COMPAS wrongly labeled Black defendants high-risk at twice the white rate.
- The bias fix is representativeness, not volume: SageMaker Clarify computes bias metrics on the raw dataset, before training, and they are model-agnostic.
- Working AI is not automatically trustworthy. Radiologists caught 46 percent of cancers unaided but only 21 percent when a faulty automated aid stayed silent.
- The EU AI Act, in force since 1 August 2024, phases in through 2026 and 2027 with fines reaching EUR 35 million or 7 percent of turnover and human oversight required for high-risk uses.
- GPT-4 scored ahead of 99 percent of people on the Torrance Tests of Creative Thinking in 2023, so the honest creativity limit is narrow competence and reasoning gaps.
Between the do-everything pitch and the hype-collapse narrative sits a question neither side answers: where exactly do AI and machine learning stop, and with which tool? This companion to the Whiteboard Architect episode The Honest Limits of AI and ML walks the same ten chapters with its fact-check citations and the tables a video cannot hold. Every number traces to a checked reference; where the record cuts against a popular claim, the record wins.

The data decides
No model has direct experience of the world; what it knows came in through its training data. Feed it incorrect, misleading, or unrepresentative examples and the output turns inaccurate, sometimes discriminatory. Wikipedia’s algorithmic-bias article tracks the mechanism: bias is learned, amplifies as training proceeds, and can leave a deployed system digitizing cultural prejudice. Poor-quality data in, poor-quality model out; imbalanced data is the quieter failure beside it.
Imbalance works by majority rule: the dominant class is served well, the minority poorly. Not one of the four cases was designed to discriminate; the data taught it.
| Case | Year | What the data did |
|---|---|---|
| Amazon resume screener | 2015 | Penalized the word "women’s", excluded all-women’s colleges; Amazon switched it off |
| COMPAS recidivism scores | 2016 analysis | Black defendants wrongly labeled high-risk at twice the white rate (ProPublica) |
| Mortgage algorithms | 2019 | UC Berkeley found lending disparities hitting Latino and African American borrowers |
| Facial recognition arrests | ongoing | Misidentifying darker-skinned faces linked to wrongful arrests of Black men |
Which named biases live inside machine learning?
The sourced taxonomy splits bias two ways. One level up, Wikipedia groups causes as pre-existing (bias that exists in society before any model touches it), technical (bias introduced by tooling and sampling), and emergent (bias that appears after deployment or in new societal contexts). Inside machine learning itself, the article names five: data bias, label bias, measurement bias, then algorithmic and deployment bias, with selection bias also named; a 2021 survey adds historical, representation, and measurement for overlapping phenomena.
The practical use of the list is diagnostic: when a system misfires, the fix differs depending on whether the damage entered through the data, the labels, the measurement, or the deployment.
Fixing bias without fooling yourself
Most checklists say collect more data. The checked sources say the dial is representativeness: larger biased datasets stay biased, and bigger samples can disregard underrepresented groups while looking more rigorous. What holds up is more representative, higher-quality data, audited for disparate error rates across every group the model will score.
Two habits keep the fix honest. Test before training: bias metrics need no trained model, they run on the raw dataset itself, so a representativeness hole is visible before any budget is spent. Then keep monitoring, because data evolves, drift appears, and retraining follows. AWS SageMaker Clarify implements the workflow end to end.
How do you measure bias before any model exists?
Clarify approaches bias as a dataset property first. Its metrics run on the rows and labels you already have, with no model in the loop, so whatever they say holds equally for every model you might build afterwards. The opening metric, facet imbalance, weighs how much training data each facet holds relative to the others; the rest look at how outcome labels spread across those same facets.
The age example shows the stakes: declare middle-aged and everyone else the two facets, and a lending dataset thin outside the middle-aged group handicaps those applicants no matter which algorithm trains on it. Read that way, a failed bias metric is cheap: you reject the dataset and have spent nothing on training.
The engineering limits
Even with clean data, the learning machinery fails in named ways; the four in the table cover most project post-mortems. Overfitting has the model learn the training set by heart, noise and all, then stumble on data it has never seen. Underfitting fails the other way, the model too simple for the structure. The one that never closes is generalization: nothing guarantees behavior outside the training distribution, and predictions on unseen data are the entire product.
| Failure mode | Symptom | Standard counters |
|---|---|---|
| Overfitting | Strong on training data, weak on new | Cross-validation, regularization, early stopping, pruning, dropout |
| Underfitting | Lacks capacity for the structure; high bias | Added complexity, better features |
| Off-distribution inputs | No guarantees beyond the training distribution | Treat unseen data as the real test |
| Compute cost | Exhaustive parameter search unaffordable | Narrower search; budget GPU time |
Compute is the fourth wall. GPUs displaced CPUs for training years ago, and the compute used by the largest deep learning runs grew about 300,000-fold from AlexNet in 2012 to AlphaZero in 2017. Applicability bounds the list from the other side: machine learning generalizes predictive patterns from examples, with competence that stays inside well-defined tasks; it is a tool for pattern-shaped problems, and a problem with no stable pattern to find is not an ML problem.
Where does the 300,000-fold compute number come from?
The figure comes from an OpenAI analysis of the field’s landmark runs: measuring AlexNet in 2012 against AlphaZero in 2017, it put the compute behind the biggest deep learning projects at 300,000 times more, doubling roughly every 3.4 months. Wikipedia’s machine learning article carries the estimate with links to the underlying analyses.
Two consequences follow for ordinary teams. Landmark training runs sit on a cost curve that doubles faster than hardware budgets, and the curve explains why exhaustive parameter search, the textbook’s tidy answer to tuning, is often the first thing a real project abandons. Specialized hardware, GPUs and the TPUs and NPUs built specifically for these workloads, exists because the arithmetic of training grew faster than general-purpose CPUs could follow.
The trust limits

A working model can remain a black box: some systems output decisions that even the team that built them cannot audit, and deep learning is naturally opaque. The toolbox for opening it includes feature importance, partial dependence plots, SHAP, and LIME. The stakes are highest in medicine, defense, finance, and law.
The other trust limit sits with the operator. Automation bias is the documented human propensity to follow automated suggestions and screen out contradictory information, even correct information. The commission version has pilots shutting down a healthy engine on a false automated alert; the omission version appears in the mammography numbers, where unaided readers caught 46 percent of the study’s cancers and only 21 percent once a faulty aid flagged nothing. Add complacency, which NASA’s aviation safety reporting glossary defines as “self-satisfaction that may result in non-vigilance”, and systems so reliable they interfere with an operator’s ability to take back control.
What are commission and omission errors, concretely?
Commission errors are acts taken on an automated directive that contradicts other evidence. The aviation case on Wikipedia’s automation-bias page: crews received false alerts instructing them to shut down an engine, and they shut down the healthy one. The automation was wrong, the contradicting instruments were right, and the automated suggestion won anyway.
Omission errors are the mirror image: something real goes unflagged by the automation, and the human, trained into reliance on the flag, misses it. In the mammography study the page cites, unaided readers caught 46 percent of the cancers; when a faulty aid flagged none, detection fell to 21 percent. The two directions differ, commission acts wrongly and omission fails to act, and both grow with how reliable the aid usually is, which is why high reliability itself is listed as a contributing factor.
The human limits
The hardest limits concern who answers when a decision touches a person’s life. Microsoft’s Responsible AI standard organizes them as six principles, better read as obligations than slogans:
| Principle | What it asks in practice |
|---|---|
| Fairness | Equitable benefits and harms, verified by measured outcomes |
| Reliability and safety | Defined behavior under defined conditions, including failures |
| Privacy and security | Transparent personal-data handling, consumer controls, differential privacy |
| Inclusiveness | Designed for people the data underrepresents |
| Transparency | Decisions legible to affected people; GDPR right to explanation |
| Accountability | Designers and deployers answer for it |
Accountability is where the principles meet enforcement. The standard’s line is that an AI system should never be the last word on a decision affecting someone’s life, and the EU AI Act gives that idea teeth: recruitment, health, justice, and law-enforcement uses are high-risk, with human-oversight obligations keeping a person in the loop by law. On jobs, three institutions span 47 percent of US employment at risk, 9 percent, and net creation.
The gravest human limit sits outside any commercial roadmap: autonomous weapons, which the US DoD Directive 3000.09 defines as systems that “select and engage targets without further intervention by a human operator” once activated. States have discussed them under the Convention on Certain Conventional Weapons since 2014, and there is still no binding prohibition, leaving blurred who answers for a particular killing.
How big is the AI jobs effect, really?
The spread across three credible institutions is the finding. Frey and Osborne’s Oxford study estimated 47 percent of US employment at high risk of computerization, and it set the terms of a decade of debate. The OECD’s 2016 study, counting across 21 member countries and measuring tasks rather than whole jobs, found on average 9 percent of jobs in foreseeable danger. The World Bank’s position is that technology destroys and creates jobs together and expects net creation over the period it models.
All three agree work changes substantially; they disagree about displacement, by a factor of five at the extremes, which is the honest number to quote when someone asks whether AI takes jobs.
The frontier limits
Three failure modes belong to this generation of large models. Hallucination: LLM chatbots embed plausible-sounding falsehoods in generated content, fabricated citations the signature case, and mitigation combines retrieval grounding, citation checking, and review by humans. The second mode is adversarial: classifiers read pixel patterns imperceptible to people; one adversarially chosen pixel flips a classification, and a planted backdoor can sit undetected. Model collapse: training on uncurated synthetic data degrades the outputs, nicknamed AI inbreeding, long-term extent unclear.
In 2023, GPT-4 scored ahead of 99 percent of people on the Torrance Tests of Creative Thinking, so the claim that machines cannot create is years stale. The verifiable limits: competence stays inside well-defined tasks, foundation models fall short on advanced reasoning and planning, and AGI itself remains hypothetical and contested.
Why can one changed pixel flip a classifier?
Image classifiers read statistics of pixel patterns, not objects the way humans see them. An adversarial example adds a perturbation tuned to those statistics, imperceptible or invisible to a person, that moves the input across a decision boundary; research cited on Wikipedia’s machine learning article includes cases where a single adversarially chosen pixel changes the predicted class.
Backdoors extend the idea during training: a model can learn a trigger pattern that forces a chosen output while behaving normally on ordinary test data, which is what makes them hard to detect. The engineering implication is that accuracy on a clean test set is an upper bound measured in the wrong environment; deployed vision systems get inputs that were never in the set, some of them crafted.
The law moves slower, but it moves

Regulation arrived on a schedule you can put on one line. The EU AI Act was proposed in April 2021, entered into force on 1 August 2024, and phases in through 2026 and 2027. It sorts uses into four risk tiers, bans social scoring and real-time biometric identification in public outright, and was revised mid-course to cover generative AI. Fines for the banned practices run to EUR 35,000,000, or to 7 percent of the offender’s total worldwide annual turnover when that is the larger figure.
Copyright runs in parallel. The US Copyright Office holds that works not created by a human author get no copyright, and a fully AI-generated image has been refused registration on that basis. 2025 guidance protects AI-assisted works where human creativity is evident, and training-data lawsuits are live: rules still being written while cases are heard.
What do the four EU AI Act risk tiers mean?
The unacceptable tier is banned outright: real-time remote biometric identification in public spaces and social scoring are the named prohibitions. The high-risk tier covers AI in health, law enforcement, justice, recruitment, education, and critical infrastructure; systems there face obligations on data quality, transparency, and human oversight, which is the clause that turns the ethics-principles discussion above into auditable requirements.
The limited-risk tier imposes transparency duties, mainly telling people when they are interacting with AI or seeing manipulated content. The minimal-risk tier covers everything else, from spam filters to game AI, and stays free of new obligations. The tier structure is why the same technology can be unregulated in one product and heavily regulated in another: the risk lives in the use, not only in the model.
The speculative end, clearly labeled
One entry belongs in its own box: the self-improving AI that becomes hard to stop. The category is hypothesized and contested, a debate about AGI-era capability, and nothing to engineer around today. A 2022 expert survey put the median extinction estimate at 5 to 10 percent. The researcher split is real and named: Hinton and Bengio worry publicly, LeCun and Ng dismiss the concern, and Gebru and Bender argue the fixation distracts from present-day harms.
Gotchas to avoid
| If you hear | The checked version |
|---|---|
| "More data fixes bias." | Representativeness fixes it; a bigger biased dataset stays biased |
| "AI can’t be creative." | Retired by the 2023 Torrance results; the accurate phrasing is narrow competence with reasoning gaps |
| "Self-improving AI is a current challenge." | It is the contested, speculative end; survey medians 5 to 10 percent |
| "The AI failed." | Audit the operator path too: reliable machine output, and a human who stopped verifying |
In short

Mapped honestly, the limits of AI and ML come down to this: what the training data carried decides what the model can do, and representativeness, collected group by group, is the bias repair that holds. Engineering bounds are just as real, from memorized training sets to compute bills that rule out exhaustive search, with opacity and operator drift layered on top of systems that technically work. EU oversight now keeps a person on high-stakes decisions, the frontier’s three failure modes are live, and the entry about self-improving AI stays where it belongs, in the labeled speculative box.
Continue learning
- Machine Learning Applications: Where ML Earns Its Keep: the previous episode’s map of where ML genuinely works.
- Narrow AI to Superintelligence: The Lines That Matter: the competence ladder behind narrow versus general.
- Machine Learning: The Honest First Lesson: the learning loop and the four model families.