Introduction to Machine Learning

The cover image is an AI-generated illustration about understanding machine learning, created using ChatGPT with the prompt: “A hand-drawn 2D animation in the Studio Ghibli style showcasing an elderly, animated scientist with wild white hair and glasses, seated in a cozy, rustic classroom. In the background, a chalkboard labeled ‘MACHINE LEARNING’ is accompanied by a cute robot diagram, while elements like a vintage computer, natural light from a wooden-framed window, and earthy tones of the room’s structure enhance the warm ambiance of the setting.” 。

Introduction

Machine Learning (ML) is a subfield of Artificial Intelligence (AI) that enables computers to learn from data and make predictions or decisions without being explicitly programmed step by step. In traditional data analysis and programming, developers often need to manually write specific rules for each analytical task. In contrast, machine learning relies on data and uses trained models to automatically identify rules or discover patterns.

With the rise of the digital age, massive amounts of information and data are generated and collected every day. Regardless of whether these datasets are inherently valuable or contain subtle, hidden insights, machine learning allows us to uncover and leverage this information. ML technologies are now widely applied across various industries, including finance, healthcare, manufacturing, and autonomous driving.

In this article, we will introduce the basic categories of machine learning methods, including supervised learning, unsupervised learning, reinforcement learning, semi-supervised learning, and active learning.

Image source

Supervised Learning

Supervised learning refers to learning patterns from labeled data to build a model that produces corresponding predicted outputs for given inputs. The labeling process is typically done manually, hence the term “supervised.” In a labeled dataset, each training example is paired with a corresponding label. The goal of the model is to learn how to predict the label based on input features. Essentially, supervised learning enables a machine to discover rules for mapping inputs to outputs.

The complete set of all possible inputs and outputs are referred to as the input space and output space, respectively. These spaces can consist of any kind of elements, even the entire Euclidean space. However, in most cases, the output space is significantly smaller than the input space.

$$ \text{Output Space} \quad << \quad \text{Input Space}. $$

Process

$$ [\text{Input Data (with labels)}] \rightarrow [\text{Model Training}] \rightarrow [\text{Predicted Output}] \rightarrow [\text{Error Calculation}] \rightarrow [\text{Model Adjustment}] \rightarrow [\text{Final Prediction}] $$

Applications

Supervised learning is widely used in classification and regression problems. For instance, in email classification, the model learns to categorize emails as either “spam” or “not spam.” In the medical field, it can be used to predict whether a patient has a certain disease based on their examination data.

Advantages and Disadvantages

• Advantages:

  • High accuracy due to training with clearly defined target labels.

• Disadvantages:

  • Requires a large amount of labeled data, and the quality of the training data directly affects the outcome.

Unsupervised Learning

Unsupervised learning, also known as unsupervised machine learning, differs from supervised learning in that it does not require labeled data. Instead, it uncovers hidden structures, patterns, or regularities within unlabeled datasets to analyze categories, transformations, or probabilities of the input data.

Similar to supervised learning, all possible inputs and outputs in unsupervised learning belong to the input space and output space, respectively. Each output is an analytical result of the input data and can represent a category, transformation, or probability.

When building an unsupervised learning model, the set of all possible models forms the hypothesis space, and the goal is to select the best model according to a given evaluation criterion.

Common methods in unsupervised learning include clustering and dimensionality reduction. Clustering is the process of grouping similar data points together. Popular techniques include k-means clustering, hierarchical clustering, and DBSCAN. Dimensionality reduction focuses on reducing the number of variables in the data while preserving important information, with methods like Principal Component Analysis (PCA) and t-SNE (t-Distributed Stochastic Neighbor Embedding).

Process

$$ [\text{Input Data (Unlabeled)}] \rightarrow [\text{Model Training}] \rightarrow [\text{Pattern Discovery}] \rightarrow [\text{Output (e.g., Clusters or Reduced Dimensions)}] $$

Applications

Unsupervised learning is often used for data exploration—such as segmenting customers into different groups using clustering techniques, or identifying underlying structures within large datasets.

Advantages and Disadvantages

• Advantages:

  • No need for labeled data, making it easier to apply to a wide range of datasets.

• Disadvantages:

  • The results are often less clear and require additional post-processing or interpretation to be useful.

Reinforcement Learning

Reinforcement Learning is a feedback-based learning method in which a model interacts with an environment and learns to adjust its behavior based on the feedback it receives. Through a process of continuous trial and error, the model learns the optimal strategy to maximize long-term cumulative rewards. These feedback signals can be positive (rewards), such as points for achieving a goal, or negative (penalties), such as losses from poor decisions.

Unlike supervised learning, reinforcement learning does not rely on labeled input-output pairs, nor does it require explicit instruction on the correct action. Instead, it focuses on balancing exploration (trying unknown actions) and exploitation (leveraging known strategies) to continuously improve decision-making. This makes reinforcement learning a fundamental machine learning approach alongside supervised and unsupervised learning.

Process

$$ [\text{Environment Interaction}] \rightarrow [\text{Action Selection}] \rightarrow [\text{Receive Feedback}] \rightarrow [\text{Trial-and-Error Strategy Adjustment}] \rightarrow [\text{Policy Update}] \rightarrow [\text{Next Action Selection}] \rightarrow [\text{Repeat Learning}] $$

Applications

Reinforcement learning is widely used in fields such as autonomous driving, robotics control, and gaming. Notable examples include AlphaGo and navigation systems in self-driving vehicles.

Advantages and Disadvantages

• Advantages:

  • Capable of solving complex decision-making problems and adapting through self-learning.

• Disadvantages:

  • Requires significant computational resources, long training times, and feedback signals that can sometimes be difficult to obtain.

Semi-Supervised Learning

Semi-supervised learning lies between supervised and unsupervised learning. This approach uses a small amount of labeled data combined with a large amount of unlabeled data for training. Since labeling data requires significant manual effort and time, while unlabeled data is easier and cheaper to obtain, semi-supervised learning offers a practical solution. By leveraging the information in the unlabeled data to assist in the supervised learning process, it allows for improved performance at a lower cost. As a result, semi-supervised learning is conceptually closer to supervised learning.

Process

$$ [\text{Labeled + Unlabeled Data}] \rightarrow [\text{Initial Model Training}] \rightarrow [\text{Predict Labels for Unlabeled Data}] \rightarrow [\text{Expand Labeled Dataset}] \rightarrow [\text{Retrain Model}] \rightarrow [\text{Final Model}] $$

Applications

Semi-supervised learning is often used in domains where labeling is difficult or costly. A common example is image recognition, where labeling a large number of images is labor-intensive. Semi-supervised learning can significantly boost model performance in such scenarios.

Advantages and Disadvantages

• Advantages:

  • Reduces the need for labeled data, lowering annotation costs.

• Disadvantages:

  • A major challenge is selecting and utilizing the most informative unlabeled data effectively during the learning process.

Active Learning

Active learning is a specialized form of semi-supervised learning in which the model actively selects the most informative unlabeled data samples and requests human annotations for them. These labeled samples are then used to improve the model. By focusing on the most valuable data, active learning aims to achieve high performance with the least amount of labeled data.

In traditional supervised learning, labeled data is typically chosen at random, this is known as passive learning. In contrast, active learning seeks to identify and prioritize the samples that would most benefit the model if labeled, thus optimizing learning outcomes while minimizing annotation costs. Like semi-supervised learning, active learning is conceptually closer to supervised learning.

Process

$$ [\text{Unlabeled Data}] \rightarrow [\text{Model Selects Valuable Samples}] \rightarrow [\text{Human Annotation}] \rightarrow [\text{Model Training}] \rightarrow [\text{Model Update}] \rightarrow [\text{Iterative Learning}] $$

Applications

Active learning is especially useful in scenarios where labeling is expensive or time-consuming and it’s not feasible to label all data. For example, in medical image analysis, active learning allows experts to focus their labeling efforts on the most challenging and informative images.

Advantages and Disadvantages

• Advantages:

  • Improves annotation efficiency and reduces labeling costs.

• Disadvantages:

  • Requires the model to effectively identify the most informative samples, which adds complexity to algorithm design.

Conclusion

The development and application of machine learning are continuously transforming various industries. From supervised learning to other types of machine learning methods, each approach has its unique use cases, advantages, and limitations. Understanding the differences among these learning paradigms helps us choose the most appropriate solution for real-world problems.

As technology continues to evolve, we are likely to see more hybrid learning models that combine multiple methods. These integrated approaches will play an increasingly important role in solving more complex and challenging problems in the future.

References

• 李航.(2022). 機器學習聖經：最完整的統計學習方法 (First Edition). 深智數位股份有限公司.

• Euclidean Space (February 13, 2025). Wikipedia, The Free Encyclopedia. Retrieved April 3, 2025, from https://en.wikipedia.org/wiki/Euclidean_space

• Reinforcement Learning (March 31, 2025). Wikipedia, The Free Encyclopedia. Retrieved April 3, 2025, from https://en.wikipedia.org/wiki/Reinforcement_learning