Neural Network Projects & Tuning

Reading about neurons, loss functions, and optimizers is only half the story. This chapter takes a real classification problem from a blank VS Code window all the way to a deployed Streamlit web app, then does the same for a regression problem. Along the way you will learn how to transform features for an ANN, how to structure a training loop, and how to choose the number of hidden layers and neurons.

A Classification Problem Statement and Setting Up VS Code

Our project: the bank wants a model that predicts whether a customer will churn — leave the bank within the next quarter — given age, balance, number of products, and a few more features. It is a binary classification task with mixed numeric and categorical columns, which makes it a realistic test of the whole pipeline.

1. Create a folder named churn-app and open it in VS Code.
2. Create a virtual environment and install tensorflow, scikit-learn, pandas, numpy, and streamlit.
3. Add churn.csv inside the project and verify it loads with pd.read_csv.

Feature Transformation Using sklearn with ANN

Neural networks need numeric inputs of comparable scale, and categorical labels need encoding. scikit-learn provides transformers that you can fit on the training split only — never on the test split — to avoid leaking information.

from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.model_selection import train_test_split

X = df.drop("Exited", axis=1)
y = df["Exited"]

pre = ColumnTransformer([
    ("num", StandardScaler(), ["Age", "Balance", "NumOfProducts"]),
    ("cat", OneHotEncoder(), ["Geography", "Gender"]),
])
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42)
X_train = pre.fit_transform(X_train)
X_test = pre.transform(X_test)

Standardizing keeps the gradients well behaved, and one-hot encoding turns categorical choices into columns the network can multiply by weights.

Step-by-Step Training with an ANN Using Optimizers and Loss Functions

1. Build a Sequential model with an input layer matching the transformed column count, one or two ReLU hidden layers, and a single sigmoid output.
2. Compile with binary_crossentropy, the Adam optimizer, and accuracy as a metric.
3. Fit for 30 epochs with a batch size of 32 and a validation split.

model = keras.Sequential([
    keras.layers.Input(shape=(X_train.shape[1],)),
    keras.layers.Dense(16, activation="relu"),
    keras.layers.Dense(8, activation="relu"),
    keras.layers.Dense(1, activation="sigmoid"),
])
model.compile(optimizer="adam", loss="binary_crossentropy",
              metrics=["accuracy"])
history = model.fit(X_train, y_train, epochs=30, batch_size=32,
                    validation_split=0.2)

Predictions with a Trained ANN Model

The sigmoid output is a probability. Convert it to a class with a threshold (usually 0.5), then evaluate with accuracy, precision, recall, and the confusion matrix. Compare your ANN against a logistic regression baseline — the ANN should match or beat it on this dataset.

import numpy as np
probs = model.predict(X_test)[:, 0]
preds = (probs >= 0.5).astype(int)
acc = (preds == y_test).mean()

Integrating the ANN Model with a Streamlit Web App

Save the model and the fitted transformer with model.save and joblib.dump, then build a small Streamlit app that loads them and makes a prediction from user inputs.

import streamlit as st
import numpy as np, joblib
from tensorflow import keras

model = keras.models.load_model("model.h5")
pre = joblib.load("pre.joblib")

st.title("Customer Churn Predictor")
age = st.slider("Age", 18, 95, 40)
balance = st.number_input("Balance", 0.0, 250000.0, 50000.0)
geo = st.selectbox("Geography", ["France", "Spain", "Germany"])

if st.button("Predict"):
    row = pre.transform([[age, balance, 1, geo, "Male"]])
    p = float(model.predict(row)[0, 0])
    st.success(f"Churn probability: {p:.1%}")
    st.warning("High risk") if p > 0.5 else st.info("Low risk")

Deploying the Streamlit Web App with the ANN Model

To put the app online: create requirements.txt listing streamlit, tensorflow, joblib, pandas, and scikit-learn; push the folder to a GitHub repository; then connect the repository to Streamlit Community Cloud and set the main entry file to app.py. Every push to the repo redeploys automatically.

ANN Regression Practical Implementation

The same recipe works for regression with three changes: no activation on the output neuron, mean_squared_error as the loss, and metrics like MAE instead of accuracy. Here we predict house prices from square footage, number of bedrooms, and location:

model = keras.Sequential([
    keras.layers.Input(shape=(X_train.shape[1],)),
    keras.layers.Dense(32, activation="relu"),
    keras.layers.Dense(16, activation="relu"),
    keras.layers.Dense(1),            # no activation: raw number
])
model.compile(optimizer="adam", loss="mse", metrics=["mae"])

Scale the target too (or use log1p) when prices span several orders of magnitude, and always compare predictions against the scale of the original data.

Finding Optimal Hidden Layers and Hidden Neurons

There is no formula that returns "the best architecture", but a pragmatic search works well:

• Start small — one hidden layer often suffices for tabular data; add a second if the first underfits.
• Grow neurons until validation accuracy stops improving, then stop. Common starting points: between the input size and twice the input size.
• Treat model creation as a function and sweep it with for loops over layer counts and neuron counts, recording validation scores.
• Add dropout and early stopping last; they trade a little training accuracy for better generalization.

def build_model(n_layers, n_neurons):
    model = keras.Sequential()
    model.add(keras.layers.Input(shape=(X_train.shape[1],)))
    for _ in range(n_layers):
        model.add(keras.layers.Dense(n_neurons, activation="relu"))
    model.add(keras.layers.Dense(1, activation="sigmoid"))
    model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
    return model