Articles
Posts Tags Categories Random
Partners
About
About Author Portfolio Terms of Use Privacy Policy Time Travel Donate
More
Settings Funny Tools Useful Websites Student Works Message Board Recent Comments

Contents

Introduction to Perceptron

 Included in Machine Learning
 Published   Updated    813 words   4 minutes   views
https://raw.githubusercontent.com/Josh-test-lab/website-assets-repository/main/posts/Introduction%20to%20Perceptron/cover%20image.png

The cover image about perceptron was generated by ChatGPT, and it used the following prompt: “A digital illustration of a perceptron model in a 16:9 ratio. Colorful input nodes labeled x₁, x₂, x₃ with weights w₁, w₂, w₃ pointing to a large circle containing a brain icon and activation function symbols (like f and Σwᵢxᵢ). Clean, modern, educational style with an off-white background and bold title text ‘PERCEPTRON’ at the top. Suitable for a tutorial article cover.”

Introduction

The Perceptron is one of the earliest models of artificial neural networks and laid the foundation for modern machine learning, often regarded as the “ancestor” of neural networks. It performs simple binary classification tasks and mathematically simulates the logic of how neurons operate. Even today, it is frequently used as a teaching example for beginners in machine learning.

History

The model’s origins trace back to 1943 when American neuroscientist Warren Sturgis McCulloch and logician Walter Pitts proposed a mathematical model of artificial neurons. In 1949, Canadian psychologist Donald Hebb introduced a learning rule for neurons—Hebb’s Rule. Then in 1957, American neurobiologist Frank Rosenblatt developed the Perceptron, a machine designed to simulate human perceptual ability. He successfully implemented it at the Cornell Aeronautical Laboratory and in 1959 built a hardware device called Mark I Perceptron, used for recognizing English letters.

Initially, the Perceptron was seen as a model with great potential. It became the first type of feedforward neural network capable of performing binary classification. However, it was later discovered that it could only solve linearly separable problems.

Working Principle

A perceptron usually consists of input and output nodes. Each input node $x_i$ is multiplied by a corresponding weight $w_i$ and then passed to the output node.

$$ input \ x_i \rightarrow w_i \cdot x_i \rightarrow output $$

More complex perceptrons have more input and output nodes. Each input $x_i$ is multiplied by its corresponding weight $w_i$ and the weighted sum is calculated. If the sum exceeds a certain threshold value $t$, the output is 1; otherwise, the output is 0. This is the simplest way to perform classification using a perceptron.

$$ input \ x_i \rightarrow \left\{\begin{matrix} 1, \text{ if } \sum_i w_i x_i > t; \\ 0, \text{ if } \sum_i w_i x_i \leq t, \end{matrix}\right. \quad \text{ where } t \text{ is the threshold} \rightarrow output $$

In the above example, the output is not limited to $\{0, 1\}$, it can also be $\{-1, 1\}$ , depending on the specific application.

We can also express a perceptron using vector notation. Suppose $\mathbf{x} \in X \subseteq \mathbb{R}^n$ is an input vector, $\mathbf{y} \in Y = \{-1, 1\}$ is the output, $\mathbf{w} \in \mathbb{R}^n$ is the weight vector, and $b \in \mathbb{R}$ is the bias. Then the perceptron model can be represented as:

$$ f(\mathbf{x}) = sign(\mathbf{w} \cdot \mathbf{x} + b), $$

where $\mathbf{w} \cdot \mathbf{x}$ is the dot product of $\mathbf{w}$ and $\mathbf{x}$, and $sign(z)$ is the sign function, ensuring the output lies in the space $Y$.

Thus, the perceptron output becomes:

$$ y = sign(z) = \left\{\begin{matrix} 1, \text{ if } z > t; \\ -1, \text{ otherwise}, \end{matrix}\right., $$

where $t \in \mathbb{R}$ is the threshold.

Algorithm

We can also construct a Perceptron Learning Algorithm, a simple and effective way to solve binary classification problems. The algorithm proceeds as follows:

  1. Choose initial values $w_0$ and $b_0$ for the weight vector $w$ and bias $b$.
  2. For each sample $x_i$ and corresponding label $y_i$ in the training set, compute the prediction $\hat{y}_i$ using the perceptron: $$ \hat{y}_i = \text{sign}(w \cdot x_i + b) $$
  3. Compute the loss function $L(\hat{y}_i, y_i)$ and decide whether to terminate the algorithm.
  4. Update the weight $w$ and bias $b$: $$ w \leftarrow w + \eta \cdot y_i \cdot x_i; \\ b \leftarrow b + \eta \cdot y_i, $$ where $\eta$ is the learning rate.
  5. Repeat from step 2 until all samples are correctly classified or the maximum number of iterations is reached.

This algorithm guarantees convergence to a correct separating hyperplane if the training data is linearly separable.

Conclusion

The perceptron model, with its simple linear structure and sign function, demonstrates basic binary classification capabilities. Its learning algorithm, conceptually straightforward, showcases the early learning potential of neural networks. As a milestone in the history of machine learning, the perceptron laid the groundwork for multilayer neural networks and is a precursor to modern deep learning theories. Although it was once abandoned due to its inability to solve non-linear problems, its clear mathematical logic continues to serve as an excellent starting point for understanding machine learning and neural networks.

References

Updated on 2025-06-03 23:09 UTC+0800   aa126365
Creative Commons License
 Machine Learning, Algorithm
Back | Home