What Is a Neural Network?

Analogy categoryAnalogies

A neural network is a machine learning model composed of layers of interconnected nodes, called neurons, loosely inspired by the way biological neurons transmit signals. Each neuron computes a weighted sum of its inputs, adds a bias term, and passes the result through a nonlinear activation function. By stacking many such neurons into layers, and stacking many layers on top of each other, neural networks can approximate extremely complex, nonlinear functions, which is why they underlie modern advances in image recognition, natural language processing, and many other domains.

🏏

Cricket analogy: A single fielder reacts to a ball by weighing distance, angle, and pace before deciding to dive; stack eleven such fielders across multiple lines of the field and the team can respond to almost any shot, however complex.

Layers: Input, Hidden, and Output

A typical feedforward neural network has an input layer, whose size matches the number of input features; one or more hidden layers, where the actual representation learning happens; and an output layer, whose size and activation depend on the task — a single sigmoid neuron for binary classification, a softmax layer for multi-class classification, or a single linear neuron for regression. A network with more than one hidden layer is often called a 'deep' neural network, and the field of deep learning is largely about designing and training such multi-layer architectures effectively.

🏏

Cricket analogy: A team's structure has openers matching the number of overs available (input), middle-order batters who build the actual innings (hidden layers doing the real work), and a finisher whose role depends on the format — steady accumulation for Tests, big hitting for T20; more finishing specialists in the order is like a deeper batting lineup.

Weights, Biases, and the Forward Pass

Each connection between neurons has an associated weight, and each neuron has a bias. During the forward pass, an input vector flows through the network layer by layer: at each neuron, inputs are multiplied by their corresponding weights, summed together with the bias, and passed through an activation function such as ReLU or sigmoid. The final layer's output is the network's prediction. Weights and biases start out randomly initialized and are the parameters that training adjusts — via gradient descent and backpropagation — so that the network's predictions increasingly match the true labels in the training data.

🏏

Cricket analogy: Each partnership between two batters has a run rate, and each batter has a natural risk tolerance; as the innings flows over-by-over, shots are chosen by weighing the required rate against risk and passed through a decision like 'attack' or 'defend', with the final score being the outcome.

python

import numpy as np

def relu(z):
    return np.maximum(0, z)

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

# A tiny two-layer network: 3 inputs -> 4 hidden units -> 1 output
np.random.seed(0)
W1 = np.random.randn(3, 4) * 0.1
b1 = np.zeros(4)
W2 = np.random.randn(4, 1) * 0.1
b2 = np.zeros(1)

def forward(x):
    z1 = x @ W1 + b1
    a1 = relu(z1)          # hidden layer activation
    z2 = a1 @ W2 + b2
    a2 = sigmoid(z2)        # output layer activation (binary classification)
    return a2

x_sample = np.array([[0.5, -1.2, 0.3]])
prediction = forward(x_sample)
print("Predicted probability:", prediction)

The Universal Approximation Theorem states that a feedforward network with even a single hidden layer of sufficient width can approximate any continuous function on a bounded domain to arbitrary precision — this is why neural networks are so flexible, though in practice deeper (not just wider) networks tend to learn efficient representations more effectively.

Without a nonlinear activation function, stacking multiple layers is mathematically equivalent to a single linear layer, no matter how many layers you add — the composition of linear functions is still linear. Nonlinearity (ReLU, sigmoid, tanh, etc.) is what actually gives neural networks their expressive power.

Why Depth and Nonlinearity Matter

Each layer of a neural network can be thought of as learning progressively more abstract features from the previous layer's output — in an image classifier, early layers might detect edges, middle layers might detect shapes or textures, and later layers might detect entire objects. This hierarchical feature learning, combined with nonlinear activations that let the network model complex decision boundaries, is what distinguishes neural networks from simpler linear models like logistic regression, which can only separate classes with a straight line or hyperplane.

🏏

Cricket analogy: A young player first learns to grip the bat correctly (edges), then to time a basic forward defense (shapes), and only years later reads bowlers' hands to anticipate late swing and play match-winning innings (objects) — well beyond what a beginner following a simple straight-bat rule could ever achieve.

A neural network stacks layers of neurons, each computing a weighted sum plus bias, passed through a nonlinear activation function.
Networks have an input layer, one or more hidden layers, and an output layer whose shape depends on the task.
The forward pass computes predictions by propagating inputs through the layers using the current weights and biases.
Nonlinear activation functions are essential; without them, stacked layers collapse into an equivalent single linear layer.
Deeper layers tend to learn progressively more abstract, higher-level features.
Training adjusts weights and biases via gradient descent and backpropagation to minimize prediction error.

Practice what you learned

1. What does a single neuron in a neural network compute before applying its activation function?

2. Why is a nonlinear activation function necessary between layers of a neural network?

3. What does the Universal Approximation Theorem state?

4. What determines the appropriate activation function and size of a neural network's output layer?

5. In the analogy of hierarchical feature learning in an image classifier, what do early layers typically learn compared to later layers?

Was this page helpful?

Activation Functions

Topics covered

#Python#MachineLearningBasicsStudyNotes#MachineLearning#WhatIsANeuralNetwork#Neural#Network#Layers#Input#Networking#StudyNotes#SkillVeris

What Is a Neural Network?