← Back to Tutorials Chapter 2: ML Foundations

Mathematical and Conceptual Foundations

Before building ML models, it is essential to understand the mathematical concepts that power them. This chapter covers linear algebra, calculus, probability, and the basics of neural networks and deep learning.

Linear Algebra

Linear algebra is the backbone of ML. Vectors and matrices represent data and model parameters. Operations like matrix multiplication power neural network computations. Eigenvalues and eigenvectors are used in dimensionality reduction techniques like PCA.

# Vectors and matrices in Python
import numpy as np

# Create a vector
v = np.array([1, 2, 3])

# Create a matrix
M = np.array([[1, 2], [3, 4]])

# Matrix multiplication
result = M @ M
print(result)

Calculus

Derivatives and gradients are used in optimization. Gradient descent, the most common optimization algorithm, uses derivatives to minimize loss functions. Partial derivatives and the chain rule are essential for backpropagation in neural networks.

Probability and Statistics

Probability theory helps model uncertainty. Bayes' theorem is fundamental to many ML algorithms. Concepts like distributions, expectation, variance, and conditional probability appear throughout ML.

Note: You do not need to be a mathematician to do ML, but understanding these concepts will help you debug models and choose the right algorithms.

Artificial Intelligence Basics

AI encompasses a broad range of techniques including rule-based systems, search algorithms, knowledge representation, and planning. ML is the data-driven branch of AI that has seen the most success in recent years.

Neural Networks Overview

A neural network consists of layers of interconnected neurons. Each neuron computes a weighted sum of its inputs, adds a bias, and applies an activation function. Deep neural networks have many hidden layers that learn hierarchical representations.

Activation Functions

Common activation functions include ReLU (Rectified Linear Unit), sigmoid, and tanh. ReLU is most popular for hidden layers because it avoids the vanishing gradient problem.

Deep Learning Introduction

Deep learning uses deep neural networks (many layers) to model complex patterns. It excels at image recognition, natural language processing, and game playing. Frameworks like TensorFlow and PyTorch make it accessible.

Getting Datasets

Kaggle is the most popular platform for finding datasets and competing in ML challenges. The UCI Machine Learning Repository offers classic benchmark datasets. Other sources include government open data portals and academic repositories.

Understanding Your Data

Before building models, explore your data. Check for missing values, outliers, data types, and distributions. Visualizations help uncover patterns and relationships between features.

Categorical Data Handling

Categorical data must be converted to numerical form for ML algorithms. Common techniques include one-hot encoding, label encoding, and ordinal encoding. Choice of encoding method depends on whether categories have a natural order.

# One-hot encoding with pandas
import pandas as pd

df = pd.DataFrame({
    'color': ['red', 'blue', 'green', 'blue']
})

encoded = pd.get_dummies(df, columns=['color'])
print(encoded)

Tip: Use one-hot encoding for nominal categories (no order) and ordinal encoding for categories with a natural ranking (e.g., small, medium, large).

Summary

Foundational knowledge in math, data handling, and neural network concepts will prepare you for building effective ML models. Practice these basics before moving to advanced topics.

Exercise 2.1: Download a dataset from Kaggle. Load it with pandas and identify which columns are numerical, categorical, and which have missing values. Write code to one-hot encode at least one categorical column.