Multi-dimensional Scaling (MDS) is a dimensionality reduction technique that calculates pairwise distances between objects in a dataset and represents them in a lower-dimensional space while preserving the original relative distance structure. MDS can be seen as a method for visualizing data to intuitively observe the relative relationships between data points.
In this era of big data, our lives are filled with various forms of information. If we treat each piece of data as an observation and collect them together, we can roughly obtain a dataset composed of many individual data points. These datasets can be either structured or unstructured. By organizing and filtering the data to extract useful information and eliminate irrelevant parts, we can create a dataset suitable for analysis, model training, or academic research. A dataset may include different data types such as numbers, text, images, audio, or video, depending on its intended application and the methods used to collect and organize it.
Cross-validation (CV) is a commonly used model validation technique in machine learning, employed to assess a model’s generalization ability on unseen data. By splitting the dataset into training, validation, and test sets, and repeatedly conducting training and evaluation, cross-validation can effectively test a model’s performance even when the data is limited. It helps evaluate generalization ability and is useful in reducing issues like overfitting and bias. Cross-validation plays a crucial role in controlling overfitting, selecting the best model, and tuning hyperparameters. In particular, it helps predict how a model will perform on new data, making it an indispensable part of modern machine learning workflows.
The Taiwan Computing Cloud (TWCC) is a platform under the National Center for High-Performance Computing (NCHC), offering AI technology development and cloud computing services. TWCC features rapid deployment, efficient orchestration, intelligent computing, and centralized data integration. By leveraging advanced container and GPU technologies, it enables the swift creation of secure and flexible computing environments. These capabilities support large-scale parallel processing and efficient data management, accelerating development workflows and meeting diverse application demands.
The National Center for High-performance Computing (NCHC), part of the National Applied Research Laboratories (NARLabs), is a research institution dedicated to technologies such as high-performance computing, storage, networking, and platform integration. As big data and AI become increasingly prevalent, having a solid environment for model training has become crucial. As Taiwan’s leading national lab in large-scale computing platforms and academic research network infrastructure, NCHC not only excels in cybersecurity and data encryption but is also actively developing advanced HPC techniques, quantum computing, and big data analytics. Training and deploying models on NCHC's systems ensures not only fast computation but also stability and security, making it a top choice for researchers.
