8. Challenges & Limitations of AI

Ethics & Challenges

Data Issues

• Data Quality — Garbage in, garbage out. Noisy, incomplete, or inaccurate data leads to poor models.
• Data Quantity — Many ML models require massive datasets, which may not be available for niche problems.
• Data Privacy — Collecting sufficient data may conflict with privacy regulations (GDPR, CCPA).
• Data Imbalance — When one class dominates (e.g., 95% non-fraud, 5% fraud), models may ignore the minority class.

Explainability (The Black Box Problem)

Deep neural networks with millions of parameters are difficult to interpret. When a model makes a decision, it's often unclear why. This is problematic in regulated industries like healthcare and finance where decisions require justification.

Robustness and Security

• Adversarial Attacks — Small, imperceptible perturbations can fool AI systems (e.g., changing a few pixels to make a stop sign look like a speed limit sign)
• Data Poisoning — Attackers inject malicious data during training to corrupt the model
• Model Inversion — Extracting training data from model outputs, potentially leaking sensitive information

Scalability

Training large AI models requires enormous computational resources. GPT-4's training cost is estimated at $100M+. The environmental impact of training and running large models is also a concern — training a single large model can emit as much CO₂ as several cars over their lifetimes.

Common Pitfalls

• Overfitting — Model memorizes training data instead of learning general patterns
• Underfitting — Model is too simple to capture underlying patterns
• Leakage — Training data accidentally contains information about the target that won't be available in production
• Concept Drift — Data distribution changes over time, degrading model performance
• Confirmation Bias — Selecting data or features that confirm pre-existing beliefs