文章
文章 標籤 分類
夥伴
關於
關於作者 隱私權政策
更多
有趣 工具 實用網站 留言板 時間旅行

目錄

1140422 meeting

 收錄於 Meeting
 發佈於   更新於    約 17422 字   預計閱讀 35 分鐘   次閱讀

前言

本次實驗原先欲使用環境部空氣品質即時資料進行分析與填補，奈何資料缺值過多，整理不易，不適合用於測試程式碼可行性。故而後轉移使用苗栗縣三義鄉空氣品質資料。

資料整理

以下為原先使用環境部資料進行整理的程式碼，所有資料將轉換為 .npy 檔案。

環境部即時資料

定義函數

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

# import modules
import os
import numpy as np
import pandas as pd

# functions
def detect_encoding(file_path):
    """
    Detect the encoding of a file.
    """
    import chardet

    with open(file_path, 'rb') as file:
        rawdata = file.read()
    result = chardet.detect(rawdata)
    encoding = result['encoding']
    return encoding

def turn_to_numeric(data_npy):
    ## 轉換成 numeric 或 nan
    cleaned_npy = np.full_like(data_npy, fill_value=np.nan, dtype='float')

    for idx in np.ndindex(data_npy.shape): # 遍歷所有索引
        val = data_npy[idx]
        try:
            if val is None:
                cleaned_npy[idx] = np.nan
            else:
                cleaned_npy[idx] = float(val)
        except (ValueError, TypeError):
            cleaned_npy[idx] = np.nan
    
    return cleaned_npy

def print_nan_count(cleaned_npy):
    ## 輸出個測站 NAN 數量
    for pollutant, station, in np.ndindex(cleaned_npy.shape[0:2]):
        print(f"Pollutant: {pollutant}, Station: {station}, NaN: {np.isnan(cleaned_npy[pollutant][station]).sum()}, ")

資料整理

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# main programs
## file name
file_name = '即時值查詢.csv'

## EDA
data = pd.read_csv(filepath_or_buffer = file_name,
                   skiprows = 2,
                   encoding = detect_encoding(file_name))
print(data.head())
print(f"data shape: {data.shape}")
執行結果參考
 1
 2
 3
 4
 5
 6
 7
 8
 9
10

   測站          日期  測項    00    01    02    03    04    05    06    07    08  ...    12    13    14    15    16    17    18    19    20    21    22    23
0  基隆  2024/01/01  CO  0.35  0.37  0.38  0.39  0.39  0.39  0.41   0.4  0.39  ...  0.37  0.38  0.36  0.35  0.33  0.32  0.31  0.29  0.29  0.34  0.36  0.39 
1  基隆  2024/01/02  CO  0.39  0.32  0.31  0.32   0.3  0.33  0.48  0.92  1.29  ...  0.29   0.3  0.28  0.29  0.38  0.75  0.75  0.87  0.71  0.58  0.62  0.54 
2  基隆  2024/01/03  CO  0.51  0.58  0.58   0.6  0.61  0.69  0.31  0.31  0.45  ...  0.47     #     #  0.47  0.44  0.43  0.36  0.36  0.33  0.35  0.35  0.34 
3  基隆  2024/01/04  CO  0.33  0.29  0.28  0.33  0.32  0.29  0.29  0.36  0.37  ...  0.31   0.3  0.34  0.32  0.44  0.56   0.8  0.98  1.01  0.76  0.61  0.67 
4  基隆  2024/01/05  CO  0.59  0.45  0.54  0.41  0.38  0.42  0.55  0.62  0.49  ...  0.35  0.35  0.35  0.34  0.37  0.36  0.58  0.62  0.62   0.6  0.54  0.62 

[5 rows x 27 columns]

data shape: (60290, 27)

資料分類

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

## 資料分類
stations = pd.unique(data.iloc[:, 0])
dates = sorted(pd.unique(data.iloc[:, 1]))
pollutants = pd.unique(data.iloc[:, 2])
print(f"length of data:\n  stations: {len(stations)}, dates: {len(dates)}, total times: {len(dates) * 24}, pollutants: {len(pollutants)}")

data_dict = {pollutant: {station: {date: [None for _ in range(24)] for date in dates} for station in stations} for pollutant in pollutants}

for i in range(data.shape[0]):
    station = data.iloc[i, 0]
    date = data.iloc[i, 1]
    pollutant = data.iloc[i, 2]
    data_dict[pollutant][station][date] = data.iloc[i, 3:27].values
    
## 檢視其中某項的前五筆
pollutant = pollutants[0]
print(f"Pollutant: '{pollutant}'")
for station, values in data_dict[pollutant].items():
    print(f"{station}: {values[dates[0]][:5]}")
執行結果參考
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

length of data:
  stations: 26, dates: 464, total times: 11136, pollutants: 5

基隆: ['0.35' '0.37' '0.38' '0.39' '0.39']
士林: ['0.35' '0.37' '0.37' '0.39' '0.4']
大同: ['0.48' '0.56' '0.47' '0.46' '0.45']
中山: ['0.41' '0.43' '0.42' '0.42' '0.42']
古亭: ['0.37' '0.35' '0.35' '0.37' '0.39']
松山: ['0.36' '0.37' '0.38' '0.37' '0.38']
陽明: ['0.36' '0.38' '0.39' '0.41' '0.42']
萬華: ['0.4' '0.43' '0.43' '0.42' '0.43']
三重: ['0.63' '0.72' '0.62' '0.6' '0.54']
土城: ['0.36' '0.37' '0.37' '0.37' '0.39']
永和: ['0.43' '0.4' '0.4' '0.4' '0.42']
汐止: ['0.32' '0.35' '0.35' '0.36' '0.37']
板橋: ['0.4' '0.41' '0.4' '0.41' '0.42']
林口: ['0.36' '0.37' '0.38' '0.38' '0.39']
淡水: ['0.33' '0.35' '0.37' '0.38' '0.39']
菜寮: ['0.4' '0.39' '0.4' '0.41' '0.42']
新店: ['0.35' '0.35' '0.35' '0.36' '0.37']
新莊: ['0.34' '0.36' '0.37' '0.37' '0.39']
萬里: ['0.36' '0.38' '0.39' '0.4' '0.4']
富貴角: ['0.31' '0.31' '0.33' '0.34' '0.35']
大園: ['0.35' '0.35' '0.37' '0.38' '0.4']
中壢: ['0.44' '0.47' '0.44' '0.46' '0.43']
平鎮: ['0.36' '0.39' '0.4' '0.41' '0.42']
桃園: ['0.35' '0.37' '0.4' '0.39' '0.4']
龍潭: ['0.36' '0.33' '0.33' '0.34' '0.35']
觀音: ['0.3' '0.33' '0.35' '0.35' '0.36']

轉換檔案

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

## 轉換成 np
data_npy = []
for pollutant in pollutants:
    station_list = []
    for station in stations:
        time_series = []
        for date in dates:
            time_series.extend(data_dict[pollutant][station][date])
        station_list.append(time_series)
    data_npy.append(station_list)

data_npy = np.asarray(data_npy)
print(data_npy[:, :5, :5])
print(f"data_npy shape: {data_npy.shape}")
執行結果參考
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

[[['0.35' '0.37' '0.38' '0.39' '0.39']
  ['0.35' '0.37' '0.37' '0.39' '0.4']
  ['0.48' '0.56' '0.47' '0.46' '0.45']
  ['0.41' '0.43' '0.42' '0.42' '0.42']
  ['0.37' '0.35' '0.35' '0.37' '0.39']]

 [['4.2' '3.7' '3.5' '3.8' '3.4']
  ['5.3' '5.5' '5' '4.7' '4.5']
  ['10.9' '12.9' '10.3' '8.8' '8.5']
  ['8.5' '9.2' '7.6' '6.4' '6.8']
  ['7.7' '7.5' '5.9' '5.4' '5.7']]

 [['52' '51' '49' '47' '47']
  ['52' '51' '49' '48' '47']
  ['45' '42' '43' '42' '41']
  ['49' '46' '46' '45' '43']
  ['48' '46' '47' '46' '43']]

 [['38' '36' '35' '36' '36']
  ['35' '34' '32' '30' '34']
  ['34' '33' '40' '43' '38']
  ['38' '29' '32' '35' '36']
  ['58' '36' '32' '32' '35']]

 [['1.4' '2.8' '2.3' '2.3' '2.6']
  ['1.3' '1' '0.8' '0.8' '0.9']
  ['1.1' '0.8' '0.8' '0.9' '0.8']
  ['0.7' '1' '0.9' '0.9' '1']
  ['2.1' '1.7' '0.9' '1.1' '1.2']]]

data_npy shape: (5, 26, 11136)

檔案儲存

1
2
3
4
5
6
7
8

## 儲存
np.save('data.npy', data_npy)

## 轉換成 numeric 或 nan
cleaned_npy = turn_to_numeric(data_npy)

## 輸出個測站 NAN 數量
print_nan_count(cleaned_npy)
執行結果參考
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130

Pollutant: 0, Station: 0, NaN: 176,
Pollutant: 0, Station: 1, NaN: 174,
Pollutant: 0, Station: 2, NaN: 129,
Pollutant: 0, Station: 3, NaN: 117,
Pollutant: 0, Station: 4, NaN: 133,
Pollutant: 0, Station: 5, NaN: 131,
Pollutant: 0, Station: 6, NaN: 95,
Pollutant: 0, Station: 7, NaN: 171,
Pollutant: 0, Station: 8, NaN: 85,
Pollutant: 0, Station: 9, NaN: 120,
Pollutant: 0, Station: 10, NaN: 125,
Pollutant: 0, Station: 11, NaN: 99,
Pollutant: 0, Station: 12, NaN: 96,
Pollutant: 0, Station: 13, NaN: 135,
Pollutant: 0, Station: 14, NaN: 142,
Pollutant: 0, Station: 15, NaN: 90,
Pollutant: 0, Station: 16, NaN: 119,
Pollutant: 0, Station: 17, NaN: 104,
Pollutant: 0, Station: 18, NaN: 165,
Pollutant: 0, Station: 19, NaN: 293,
Pollutant: 0, Station: 20, NaN: 120,
Pollutant: 0, Station: 21, NaN: 127,
Pollutant: 0, Station: 22, NaN: 106,
Pollutant: 0, Station: 23, NaN: 94,
Pollutant: 0, Station: 24, NaN: 134,
Pollutant: 0, Station: 25, NaN: 190,
Pollutant: 1, Station: 0, NaN: 210,
Pollutant: 1, Station: 1, NaN: 222,
Pollutant: 1, Station: 2, NaN: 201,
Pollutant: 1, Station: 3, NaN: 212,
Pollutant: 1, Station: 4, NaN: 199,
Pollutant: 1, Station: 5, NaN: 161,
Pollutant: 1, Station: 6, NaN: 140,
Pollutant: 1, Station: 7, NaN: 216,
Pollutant: 1, Station: 8, NaN: 132,
Pollutant: 1, Station: 9, NaN: 167,
Pollutant: 1, Station: 10, NaN: 169,
Pollutant: 1, Station: 11, NaN: 131,
Pollutant: 1, Station: 12, NaN: 159,
Pollutant: 1, Station: 13, NaN: 198,
Pollutant: 1, Station: 14, NaN: 195,
Pollutant: 1, Station: 15, NaN: 179,
Pollutant: 1, Station: 16, NaN: 182,
Pollutant: 1, Station: 17, NaN: 174,
Pollutant: 1, Station: 18, NaN: 142,
Pollutant: 1, Station: 19, NaN: 432,
Pollutant: 1, Station: 20, NaN: 210,
Pollutant: 1, Station: 21, NaN: 179,
Pollutant: 1, Station: 22, NaN: 248,
Pollutant: 1, Station: 23, NaN: 165,
Pollutant: 1, Station: 24, NaN: 193,
Pollutant: 1, Station: 25, NaN: 261,
Pollutant: 2, Station: 0, NaN: 213,
Pollutant: 2, Station: 1, NaN: 189,
Pollutant: 2, Station: 2, NaN: 244,
Pollutant: 2, Station: 3, NaN: 139,
Pollutant: 2, Station: 4, NaN: 141,
Pollutant: 2, Station: 5, NaN: 169,
Pollutant: 2, Station: 6, NaN: 124,
Pollutant: 2, Station: 7, NaN: 232,
Pollutant: 2, Station: 8, NaN: 151,
Pollutant: 2, Station: 9, NaN: 152,
Pollutant: 2, Station: 10, NaN: 166,
Pollutant: 2, Station: 11, NaN: 241,
Pollutant: 2, Station: 12, NaN: 198,
Pollutant: 2, Station: 13, NaN: 159,
Pollutant: 2, Station: 14, NaN: 186,
Pollutant: 2, Station: 15, NaN: 120,
Pollutant: 2, Station: 16, NaN: 156,
Pollutant: 2, Station: 17, NaN: 157,
Pollutant: 2, Station: 18, NaN: 159,
Pollutant: 2, Station: 19, NaN: 319,
Pollutant: 2, Station: 20, NaN: 154,
Pollutant: 2, Station: 21, NaN: 126,
Pollutant: 2, Station: 22, NaN: 138,
Pollutant: 2, Station: 23, NaN: 124,
Pollutant: 2, Station: 24, NaN: 150,
Pollutant: 2, Station: 25, NaN: 181,
Pollutant: 3, Station: 0, NaN: 174,
Pollutant: 3, Station: 1, NaN: 132,
Pollutant: 3, Station: 2, NaN: 113,
Pollutant: 3, Station: 3, NaN: 116,
Pollutant: 3, Station: 4, NaN: 103,
Pollutant: 3, Station: 5, NaN: 129,
Pollutant: 3, Station: 6, NaN: 155,
Pollutant: 3, Station: 7, NaN: 161,
Pollutant: 3, Station: 8, NaN: 80,
Pollutant: 3, Station: 9, NaN: 117,
Pollutant: 3, Station: 10, NaN: 91,
Pollutant: 3, Station: 11, NaN: 131,
Pollutant: 3, Station: 12, NaN: 92,
Pollutant: 3, Station: 13, NaN: 139,
Pollutant: 3, Station: 14, NaN: 181,
Pollutant: 3, Station: 15, NaN: 77,
Pollutant: 3, Station: 16, NaN: 102,
Pollutant: 3, Station: 17, NaN: 135,
Pollutant: 3, Station: 18, NaN: 147,
Pollutant: 3, Station: 19, NaN: 259,
Pollutant: 3, Station: 20, NaN: 155,
Pollutant: 3, Station: 21, NaN: 138,
Pollutant: 3, Station: 22, NaN: 111,
Pollutant: 3, Station: 23, NaN: 143,
Pollutant: 3, Station: 24, NaN: 126,
Pollutant: 3, Station: 25, NaN: 230,
Pollutant: 4, Station: 0, NaN: 414,
Pollutant: 4, Station: 1, NaN: 260,
Pollutant: 4, Station: 2, NaN: 192,
Pollutant: 4, Station: 3, NaN: 215,
Pollutant: 4, Station: 4, NaN: 501,
Pollutant: 4, Station: 5, NaN: 458,
Pollutant: 4, Station: 6, NaN: 179,
Pollutant: 4, Station: 7, NaN: 374,
Pollutant: 4, Station: 8, NaN: 119,
Pollutant: 4, Station: 9, NaN: 298,
Pollutant: 4, Station: 10, NaN: 156,
Pollutant: 4, Station: 11, NaN: 231,
Pollutant: 4, Station: 12, NaN: 149,
Pollutant: 4, Station: 13, NaN: 345,
Pollutant: 4, Station: 14, NaN: 314,
Pollutant: 4, Station: 15, NaN: 114,
Pollutant: 4, Station: 16, NaN: 182,
Pollutant: 4, Station: 17, NaN: 193,
Pollutant: 4, Station: 18, NaN: 164,
Pollutant: 4, Station: 19, NaN: 485,
Pollutant: 4, Station: 20, NaN: 147,
Pollutant: 4, Station: 21, NaN: 509,
Pollutant: 4, Station: 22, NaN: 366,
Pollutant: 4, Station: 23, NaN: 122,
Pollutant: 4, Station: 24, NaN: 246,
Pollutant: 4, Station: 25, NaN: 222,

三義資料

以下為使用三義鄉資料進行整理的程式碼，部分程式碼套用先前定義的函數。

資料整理

1
2
3
4
5
6
7
8
9

## file name
file_name = 'SanYi_training.csv'

## EDA
data = pd.read_csv(filepath_or_buffer = file_name,
                   skiprows = 0,
                   encoding = detect_encoding(file_name))
print(data.head())
print(f"data shape: {data.shape}")
執行結果參考
1
2
3
4
5
6
7
8

   PM25  PM10    CO    NO    NO2    NOx    O3  PH_RAIN  RAINFALL     RH  SO2   TEMP  UVB  WIND_DIREC  WIND_SPEED
0    20  35.0  0.12  1.21  13.27  14.48  22.8      0.0       0.0  90.51  1.5  14.77  0.0       33.16        4.24
1    16  28.0  0.11  0.98   9.17  10.15  25.7      0.0       0.0  90.25  1.4  14.91  0.0       30.16        4.36
2    16  26.0  0.33  0.78   7.98   8.76  26.9      0.0       0.0  90.24  1.5  15.03  0.0       35.15        4.27
3    15  33.0  0.32  1.08  10.51  11.58  25.4      0.0       0.0  90.41  2.3  14.95  0.0       35.30        4.01
4    13  30.0  0.32  1.07  15.44  16.51  16.4      0.0       0.0  91.48  2.4  14.72  0.0       34.57        4.13

data shape: (43752, 15)

資料分類

1
2
3
4
5
6
7
8

## 資料分類
print(f"length of data:\n  stations: 1, total times: {data.shape[0]}, pollutants: {data.shape[1]}")
pollutants = data.keys()

data_dict = {pollutant: {'station': {'date': None}} for pollutant in pollutants}

for i in range(data.shape[1]):
    data_dict[data.keys()[i]]['station']['date'] = data.iloc[:, i].values
執行結果參考
1
2

length of data:
  stations: 1, total times: 43752, pollutants: 15

轉換資料

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

## 轉換成 np
data_npy = []
for pollutant in pollutants:
    station_list = []
    for station in range(1):
        time_series = []
        for date in range(1):
            time_series.extend(data_dict[pollutant]['station']['date'])
        station_list.append(time_series)
    data_npy.append(station_list)

data_npy = np.asarray(data_npy)
print(data_npy[:, :5, :5])
print(f"data_npy shape: {data_npy.shape}")
執行結果參考
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

[[[20.   16.   16.   15.   13.  ]]

 [[35.   28.   26.   33.   30.  ]]

 [[ 0.12  0.11  0.33  0.32  0.32]]

 [[ 1.21  0.98  0.78  1.08  1.07]]

 [[13.27  9.17  7.98 10.51 15.44]]

 [[14.48 10.15  8.76 11.58 16.51]]

 [[22.8  25.7  26.9  25.4  16.4 ]]

 [[ 0.    0.    0.    0.    0.  ]]

 [[ 0.    0.    0.    0.    0.  ]]

 [[90.51 90.25 90.24 90.41 91.48]]

 [[ 1.5   1.4   1.5   2.3   2.4 ]]

 [[14.77 14.91 15.03 14.95 14.72]]

 [[ 0.    0.    0.    0.    0.  ]]

 [[33.16 30.16 35.15 35.3  34.57]]

 [[ 4.24  4.36  4.27  4.01  4.13]]]

data_npy shape: (15, 1, 43752)

檔案儲存

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

## 儲存
np.save('SanYi.npy', data_npy)

## 轉換成 numeric 或 nan
cleaned_npy = turn_to_numeric(data_npy)

## 輸出個測站 NAN 數量
print_nan_count(cleaned_npy)

## 分成訓練集和測試集
train_npy = cleaned_npy[:, :, -105:-5]
test_npy = cleaned_npy[:, :, -5:]

train_npy = train_npy.transpose(1, 2, 0)
test_npy = test_npy.transpose(1, 2, 0)
print(train_npy.shape)
print(test_npy.shape)

np.save('SanYi_train.npy', train_npy)
np.save('SanYi_test.npy', test_npy)
執行結果參考
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

Pollutant: 0, Station: 0, NaN: 0,
Pollutant: 1, Station: 0, NaN: 5,
Pollutant: 2, Station: 0, NaN: 0,
Pollutant: 3, Station: 0, NaN: 0,
Pollutant: 4, Station: 0, NaN: 0,
Pollutant: 5, Station: 0, NaN: 0,
Pollutant: 6, Station: 0, NaN: 0,
Pollutant: 7, Station: 0, NaN: 0,
Pollutant: 8, Station: 0, NaN: 0,
Pollutant: 9, Station: 0, NaN: 0,
Pollutant: 10, Station: 0, NaN: 0,
Pollutant: 11, Station: 0, NaN: 0,
Pollutant: 12, Station: 0, NaN: 0,
Pollutant: 13, Station: 0, NaN: 0,
Pollutant: 14, Station: 0, NaN: 0,

(1, 100, 15)

(1, 5, 15)

臺灣 AI 雲 TWCC 計算

Config

1140421 實驗室網路炸掉，無法連線。1140422 的結果。

SSSD_CP/configs/model.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

wavenet:
  # WaveNet model parameters
  input_channels: 15  # Number of input channels
  output_channels: 15  # Number of output channels
  residual_layers: 64  # Number of residual layers
  residual_channels: 256  # Number of channels in residual blocks
  skip_channels: 256  # Number of channels in skip connections

  # Diffusion step embedding dimensions
  diffusion_step_embed_dim_input: 128  # Input dimension
  diffusion_step_embed_dim_hidden: 512  # Middle dimension
  diffusion_step_embed_dim_output: 512  # Output dimension

  # Structured State Spaces sequence model (S4) configurations
  s4_max_sequence_length: 500  # Maximum sequence length
  s4_state_dim: 64  # State dimension
  s4_dropout: 0.0  # Dropout rate
  s4_bidirectional: true  # Whether to use bidirectional layers
  s4_use_layer_norm: true  # Whether to use layer normalization

diffusion:
  # Diffusion model parameters
  T: 200  # Number of diffusion steps
  beta_0: 0.0001  # Initial beta value
  beta_T: 0.02  # Final beta value

SSSD_CP/configs/training.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

# Training configuration
batch_size: 80  # Batch size
output_directory: "./results/checkpoint"  # Output directory for checkpoints and logs
ckpt_iter: "max"  # Checkpoint mode (max or min)
iters_per_ckpt: 100  # Checkpoint frequency (number of epochs)
iters_per_logging: 100  # Log frequency (number of iterations)
n_iters: 5000  # Maximum number of iterations
learning_rate: 0.0002  # Learning rate

# Additional training settings
only_generate_missing: true  # Generate missing values only
use_model: 2  # Model to use for training
masking: "forecast"  # Masking strategy for missing values
missing_k: 5  # Number of missing values

# Data paths
data:
  train_path: "./datasets/SanYi/SanYi_train.npy"  # Path to training data

SSSD_CP/configs/inference.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# Inference configuration
batch_size: 80  # Batch size for inference
output_directory: "./results/checkpoint"  # Output directory for inference results
ckpt_path: "./results/checkpoint"  # Path to checkpoint for inference
trials: 1 # Replications

# Additional training settings
only_generate_missing: true  # Generate missing values only
use_model: 2  # Model to use for training
masking: "forecast"  # Masking strategy for missing values
missing_k: 5  # Number of missing values

# Data paths
data:
  test_path: "./datasets/SanYi/SanYi_test.npy"  # Path to test data

TWCC 上使用的 commands 。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

# 查詢目前目錄
pwd

# 下載 Miniconda
wget -c https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

# 安裝 Miniconda
bash Miniconda3-latest-Linux-x86_64.sh -b -f -p /home/u6025091

# 確定 conda 版本 (是否安裝成功)
conda --version

# 建立 SSSD 環境，使用 Python 3.10
conda create -n sssd python=3.10 -y

# 安裝 dos2unix ，用於轉換換行字符
sudo apt-get update
sudo apt-get install dos2unix

# 轉換 `/SSSD_CP/` 下的所有檔案
find ~/SSSD_CP/ -type f -exec dos2unix {} \;

# 提升執行權限
./envs/vm/install_on_ubuntu.sh
./envs/conda/build_conda_env.sh

chmod +x /SSSD_CP/scripts/diffusion/training_job.sh

# 安裝 libcudnn.so.8 (如果有報錯且提示找不到)
sudo apt-get update
sudo apt-get install libcudnn8 libcudnn8-dev libcudnn8-doc

# 找到 libcudnn.so.8 的位置
find / -name "libcudnn.so.8" 2>/dev/null

# 將 libcudnn.so.8 加入環境變數 (使用上方找到的位置)
export LD_LIBRARY_PATH=/home/u6025091/local/envs/sssd/lib/python3.10/site-packages/nvidia/cudnn/lib:$LD_LIBRARY_PATH

# 重新載入環境變數
source ~/.bashrc

# 執行模型訓練
./scripts/diffusion/training_job.sh -m configs/model.yaml -t configs/training.yaml

計算結果

1140421 實驗室網路炸掉，無法連線。1140422 的結果。
執行結果參考
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438

(base) u6025091@jjd26etest1140422-b452q:~$ s3cmd get --recursive --verbose s3://sssds4/SSSD_CP/datasets/SanYi/ ~/SSSD_CP/datasets/SanYi/
INFO: Retrieving list of remote files for s3://sssds4/SSSD_CP/datasets/SanYi/ ...
WARNING: Found empty root object name on S3, ignoring.
INFO: Summary: 2 remote files to download
ERROR: Parameter problem: Destination must be a directory or stdout when downloading multiple sources.
(base) u6025091@jjd26etest1140422-b452q:~$ pwd
/home/u6025091
(base) u6025091@jjd26etest1140422-b452q:~$ s3cmd get --recursive --verbose s3://sssds4/SSSD_CP/datasets/SanYi/ ~/SSSD_CP/datasets/SanYi/
INFO: Retrieving list of remote files for s3://sssds4/SSSD_CP/datasets/SanYi/ ...
WARNING: Found empty root object name on S3, ignoring.
INFO: Summary: 2 remote files to download
ERROR: Parameter problem: Destination must be a directory or stdout when downloading multiple sources.
(base) u6025091@jjd26etest1140422-b452q:~$ s3cmd get --recursive --verbose s3://sssds4/SSSD_CP/datasets/SanYi/* ~/SSSD_CP/datasets/SanYi/
INFO: Retrieving list of remote files for s3://sssds4/SSSD_CP/datasets/SanYi/* ...
INFO: Summary: 0 remote files to download
(base) u6025091@jjd26etest1140422-b452q:~$ s3cmd get --recursive --verbose s3://sssds4/SSSD_CP/datasets/SanYi/*.npy ~/SSSD_CP/datasets/SanYi/
INFO: Retrieving list of remote files for s3://sssds4/SSSD_CP/datasets/SanYi/*.npy ...
INFO: Summary: 0 remote files to download
(base) u6025091@jjd26etest1140422-b452q:~$ s3cmd get --recursive --verbose s3://sssds4/SSSD_CP/datasets/SanYi/ ~/SSSD_CP/datasets/SanYi/
INFO: Retrieving list of remote files for s3://sssds4/SSSD_CP/datasets/SanYi/ ...
WARNING: Found empty root object name on S3, ignoring.
INFO: Summary: 2 remote files to download
ERROR: Parameter problem: Destination must be a directory or stdout when downloading multiple sources.
(base) u6025091@jjd26etest1140422-b452q:~$ s3cmd get --recursive --verbose s3://sssds4/SSSD_CP/datasets/SanYi/ ~/SSSD_CP/datasets/SanYi/
INFO: Retrieving list of remote files for s3://sssds4/SSSD_CP/datasets/SanYi/ ...
WARNING: Found empty root object name on S3, ignoring.
INFO: Summary: 2 remote files to download
download: 's3://sssds4/SSSD_CP/datasets/SanYi/SanYi_test.npy' -> '/home/u6025091/SSSD_CP/datasets/SanYi/SanYi_test.npy'  [1 of 2]
 12728 of 12728   100% in    0s    53.88 KB/s  done
download: 's3://sssds4/SSSD_CP/datasets/SanYi/SanYi_train.npy' -> '/home/u6025091/SSSD_CP/datasets/SanYi/SanYi_train.npy'  [2 of 2]
 60128 of 60128   100% in    0s   222.61 KB/s  done
(base) u6025091@jjd26etest1140422-b452q:~$ conda envs list
usage: conda [-h] [-v] [--no-plugins] [-V] COMMAND ...
conda: error: argument COMMAND: invalid choice: 'envs' (choose from activate, clean, commands, compare, config, content-trust, create, deactivate, doctor, env, export, info, init, install, list, notices, package, remove, rename, repoquery, run, search, tos, uninstall, update, upgrade)
(base) u6025091@jjd26etest1140422-b452q:~$ conda env list

# conda environments:
#
                       /home/u6025091
base                 * /home/u6025091/local
sssd                   /home/u6025091/local/envs/sssd

(base) u6025091@jjd26etest1140422-b452q:~$ deactivate
DeprecationWarning: 'source deactivate' is deprecated. Use 'conda deactivate'.
(base) u6025091@jjd26etest1140422-b452q:~$ conda deactivate
u6025091@jjd26etest1140422-b452q:~$ conda activate sssd
(sssd) u6025091@jjd26etest1140422-b452q:~$ ./scripts/diffusion/training_job.sh -m configs/model.yaml -t configs/training.yaml
bash: ./scripts/diffusion/training_job.sh: No such file or directory
(sssd) u6025091@jjd26etest1140422-b452q:~$ pwd
/home/u6025091
(sssd) u6025091@jjd26etest1140422-b452q:~$ cd SSSD_CP
(sssd) u6025091@jjd26etest1140422-b452q:~/SSSD_CP$ ./scripts/diffusion/training_job.sh -m configs/model.yaml -t configs/training.yaml
Intializing conda
Activating Conda Env: sssd
[Execution - Training]
/home/u6025091/SSSD_CP/scripts/diffusion/train.py --model_config configs/model.yaml --training_config configs/training.yaml
Traceback (most recent call last):
  File "/home/u6025091/SSSD_CP/scripts/diffusion/train.py", line 5, in <module>
    import torch
  File "/home/u6025091/.local/lib/python3.10/site-packages/torch/__init__.py", line 237, in <module>
    from torch._C import *  # noqa: F403
ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory
(sssd) u6025091@jjd26etest1140422-b452q:~/SSSD_CP$ find ~/SSSD_CP/ -type f -exec dos2unix {} \;
dos2unix: converting file /home/u6025091/SSSD_CP/tests/inference/test_inference_utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/core/__init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/core/layers/test_layers_activation.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/core/layers/__init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/core/layers/test_layers_linear.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/core/hippo/__init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/core/hippo/test_hippo_utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/core/hippo/test_hippo_state_space.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/core/hippo/test_normal_plus_low_rank.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/core/hippo/test_hippo_kernel.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/core/test_core_utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/__init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/utils/test_utils_utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/utils/__init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/utils/test_logger.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/data/test_data_generator.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/data/__init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/data/test_data_dataloader.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/data/test_data_utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/data/test_data_dataset.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/training/test_training_utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/tests/training/__init__.py to Unix format...
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/report/plots_merged_004.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/report/plots_merged_001.png
dos2unix: converting file /home/u6025091/SSSD_CP/docs/report/sssd.drawio_bigger.xml to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/docs/report/sssd.drawio to Unix format...
dos2unix: Binary symbol 0x1C found at line 174
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/report/sssd.drawio_bigger.pdf
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/report/plots_merged_002.png
dos2unix: converting file /home/u6025091/SSSD_CP/docs/report/SSSDS4draw.xml to Unix format...
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/report/plots_merged_003.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/report/updated_architecture.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/data_plt/uncleaned_20050131_20161231/HUD VL.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/data_plt/uncleaned_20050131_20161231/DUNWOD.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/data_plt/uncleaned_20050131_20161231/CAPITL.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/data_plt/uncleaned_20050131_20161231/GENESE.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/data_plt/uncleaned_20050131_20161231/NORTH.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/data_plt/uncleaned_20050131_20161231/CENTRL.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/data_plt/uncleaned_20050131_20161231/LONGIL.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/data_plt/uncleaned_20050131_20161231/MILLWD.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/data_plt/uncleaned_20050131_20161231/MHK VL.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/data_plt/uncleaned_20050131_20161231/N.Y.C..png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/data_plt/uncleaned_20050131_20161231/WEST.png
dos2unix: converting file /home/u6025091/SSSD_CP/docs/instructions/Solar/solar.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/docs/instructions/Solar/updated_solar.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/docs/instructions/ETTm1/ettm1.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/docs/instructions/PEMS-Bay and METR-LA/README.md to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/docs/instructions/PEMS-Bay and METR-LA/feature_sample_process.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/docs/instructions/PTB-XL/ecg_data_preprocessing.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/docs/instructions/PTB-XL/clinical_ts/ecg_utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/docs/instructions/PTB-XL/clinical_ts/timeseries_utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/docs/instructions/PTB-XL/README.md to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/docs/instructions/Electricity/README.md to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/docs/instructions/Mujoco/README.md to Unix format...
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/images/docker_compose_ps.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/images/docker_compose_down.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/images/run_docker.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/images/clean_docker.png
dos2unix: Binary symbol 0x1A found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/docs/images/img_docker_compose_log.png
dos2unix: Binary symbol 0x01 found at line 1
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/datasets/SanYi/SanYi_test.npy
dos2unix: Binary symbol 0x01 found at line 1
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/datasets/SanYi/SanYi_train.npy
dos2unix: converting file /home/u6025091/SSSD_CP/.pre-commit-config.yaml to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/bin/utils.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/bin/color_map.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/bin/exit_code.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/Makefile to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/.github/workflows/docker.yaml to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/.github/workflows/test.yaml to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/.github/pull_request_template.md to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/inference/generator.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/inference/utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/inference/___init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/imputers/SSSDSAImputer.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/imputers/SSSDS4Imputer.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/imputers/DiffWaveImputer.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/imputers/___init__.py to Unix format...
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/imputers/__pycache__/DiffWaveImputer.cpython-312.pyc
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/imputers/__pycache__/SSSDS4Imputer.cpython-312.pyc
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/imputers/__pycache__/SSSDSAImputer.cpython-312.pyc
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/___init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/model_specs.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/layers/activation.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/layers/linear.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/layers/___init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/layers/s4/s4_layer.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/layers/s4/___init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/layers/s4/hippo/hippo.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/layers/s4/hippo/utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/layers/s4/hippo/___init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/layers/s4/hippo/state_space.py to Unix format...
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/layers/s4/hippo/__pycache__/state_space.cpython-312.pyc
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/layers/s4/hippo/__pycache__/utils.cpython-312.pyc
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/layers/s4/hippo/__pycache__/hippo.cpython-312.pyc
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/layers/s4/hippo/__pycache__/normal_plus_low_rank.cpython-312.pyc
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/layers/s4/hippo/normal_plus_low_rank.py to Unix format...
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/layers/s4/__pycache__/s4_layer.cpython-312.pyc
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/core/layers/layer_normalization.py to Unix format...
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/layers/__pycache__/linear.cpython-312.pyc
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/layers/__pycache__/activation.cpython-312.pyc
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/__pycache__/utils.cpython-312.pyc
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/core/__pycache__/model_specs.cpython-312.pyc
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/utils/utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/utils/___init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/utils/data_utils.py to Unix format...
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/utils/__pycache__/logger.cpython-312.pyc
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/utils/__pycache__/utils.cpython-312.pyc
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/utils/logger.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/___init__.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/data/generator.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/data/dataloader.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/data/utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/data/___init__.py to Unix format...
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/data/__pycache__/generator.cpython-312.pyc
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/data/__pycache__/utils.cpython-312.pyc
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/data/dataset.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/training/trainer.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/training/utils.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/sssd/training/___init__.py to Unix format...
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/training/__pycache__/trainer.cpython-312.pyc
dos2unix: Binary symbol 0x00 found at line 2
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/sssd/training/__pycache__/utils.cpython-312.pyc
dos2unix: converting file /home/u6025091/SSSD_CP/LICENCE to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/MAPE.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/datasets/pickle-loadfrst.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/datasets/nyiso-csv-to-pickle.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/datasets/fake_data.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/datasets/vis.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/datasets/csv-to-pickle.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/datasets/load-pickle-to-npy.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/datasets/nyiso-load-pickle-to-npy.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/normal_plus_low_rank_measures.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/compare_einsum.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/ar training test.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/vis.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/experiments/CPI.ipynb to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/notebooks/examples/training.ipynb to Unix format...
dos2unix: Binary symbol 0x00 found at line 5
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/envs/vm/cuda-keyring_1.1-1_all.deb
dos2unix: converting file /home/u6025091/SSSD_CP/envs/vm/install_on_ubuntu.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/vm/.gitignore to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/docker/Dockerfile.full to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/docker/README.md to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/jupyter/utils.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/jupyter/start_jupyter_lab.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/conda/pack_conda_env.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/conda/build_conda_env.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/conda/utils.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/conda/find_conda_env.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/conda/cauchy_mult_setup.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/conda/.gitignore to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/envs/conda/conda_env_info.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/poetry.lock to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/model.yaml to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/.ipynb_checkpoints/README-checkpoint.md to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/inference.yaml to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/training.yaml to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/README.md to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-2-mix-fine-tune-1.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-3-MILLWD.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-2-all-fine-tune-1.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-3-CAPITL.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-split-2.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-ar1.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-2-all-fine-tune-2.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDSA.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/toy_example.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-3-mix.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-3-all.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-2-mix.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-all.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-2-all.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-split.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_DiffWave.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-3-NYC.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/configs/legacy/config_SSSDS4-NYISO-2-mix-fine-tune-2.json to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/Dockerfile to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/README.md to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/pyproject.toml to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/.gitignore to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/services/jupyter-docker-compose.yaml to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/services/diffusion-docker-compose.yaml to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/scripts/docker/run_diffusion.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/scripts/docker/run_notebook.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/scripts/diffusion/inference_job.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/scripts/diffusion/.ipynb_checkpoints/training_job-checkpoint.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/scripts/diffusion/infer.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/scripts/diffusion/train.py to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/scripts/diffusion/training_job.sh to Unix format...
dos2unix: converting file /home/u6025091/SSSD_CP/scripts/diffusion/diffusion_process.sh to Unix format...
dos2unix: Binary symbol 0x02 found at line 670
dos2unix: Skipping binary file /home/u6025091/SSSD_CP/Miniconda3-latest-Linux-x86_64.sh
(sssd) u6025091@jjd26etest1140422-b452q:~/SSSD_CP$ ./scripts/diffusion/training_job.sh -m configs/model.yaml -t configs/training.yaml
Intializing conda
Activating Conda Env: sssd
[Execution - Training]
/home/u6025091/SSSD_CP/scripts/diffusion/train.py --model_config configs/model.yaml --training_config configs/training.yaml
Traceback (most recent call last):
  File "/home/u6025091/SSSD_CP/scripts/diffusion/train.py", line 5, in <module>
    import torch
  File "/home/u6025091/.local/lib/python3.10/site-packages/torch/__init__.py", line 237, in <module>
    from torch._C import *  # noqa: F403
ImportError: libcudnn.so.8: cannot open shared object file: No such file or directory
(sssd) u6025091@jjd26etest1140422-b452q:~/SSSD_CP$ nvidia-smi
Tue Apr 22 09:07:30 2025       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.161.08             Driver Version: 535.161.08   CUDA Version: 12.8     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  Tesla V100-SXM2-32GB           On  | 00000000:1B:00.0 Off |                    0 |
| N/A   26C    P0              42W / 300W |      0MiB / 32768MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|  No running processes found                                                           |
+---------------------------------------------------------------------------------------+
(sssd) u6025091@jjd26etest1140422-b452q:~/SSSD_CP$ find / -name "libcudnn.so.8" 2>/dev/null
/home/u6025091/local/envs/sssd/lib/python3.10/site-packages/nvidia/cudnn/lib/libcudnn.so.8
(sssd) u6025091@jjd26etest1140422-b452q:~/SSSD_CP$ export LD_LIBRARY_PATH=/home/u6025091/local/envs/sssd/lib/python3.10/site-packages/nvidia/cudnn/lib:$LD_LIBRARY_PATH
(sssd) u6025091@jjd26etest1140422-b452q:~/SSSD_CP$ source ~/.bashrc
(base) u6025091@jjd26etest1140422-b452q:~/SSSD_CP$ ./scripts/diffusion/training_job.sh -m configs/model.yaml -t configs/training.yaml
Intializing conda
Activating Conda Env: sssd
[Execution - Training]
/home/u6025091/SSSD_CP/scripts/diffusion/train.py --model_config configs/model.yaml --training_config configs/training.yaml
2025-04-22 09:08:22,366 - sssd.utils.logger - INFO - Model spec: {'wavenet': {'input_channels': 15, 'output_channels': 15, 'residual_layers': 64, 'residual_channels': 256, 'skip_channels': 256, 'diffusion_step_embed_dim_input': 128, 'diffusion_step_embed_dim_hidden': 512, 'diffusion_step_embed_dim_output': 512, 's4_max_sequence_length': 500, 's4_state_dim': 64, 's4_dropout': 0.0, 's4_bidirectional': True, 's4_use_layer_norm': True}, 'diffusion': {'T': 200, 'beta_0': 0.0001, 'beta_T': 0.02}}
2025-04-22 09:08:22,366 - sssd.utils.logger - INFO - Training spec: {'batch_size': 80, 'output_directory': './results/checkpoint', 'ckpt_iter': 'max', 'iters_per_ckpt': 100, 'iters_per_logging': 100, 'n_iters': 5000, 'learning_rate': 0.0002, 'only_generate_missing': True, 'use_model': 2, 'masking': 'forecast', 'missing_k': 5, 'data': {'train_path': './datasets/SanYi/SanYi_train.npy'}}
2025-04-22 09:08:22,377 - sssd.utils.logger - INFO - Using 1 GPUs!
2025-04-22 09:08:22,409 - sssd.utils.logger - INFO - Output directory ./results/checkpoint/T200_beta00.0001_betaT0.02
2025-04-22 09:08:40,246 - sssd.utils.logger - INFO - Current time: 2025-04-22 09:08:40
2025-04-22 09:08:43,877 - sssd.utils.logger - INFO - No valid checkpoint model found, start training from initialization.
2025-04-22 09:08:43,877 - sssd.utils.logger - INFO - Start the 1 iteration
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:04<00:00,  4.44s/it]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.66it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.80it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.77it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.80it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.81it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.53it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.81it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.81it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.81it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.80it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.80it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.84it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.80it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.81it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.77it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.84it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.84it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.80it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.84it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.81it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.59it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.84it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.84it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.80it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.79it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.81it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.84it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.84it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.78it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.80it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.81it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.82it/s]





100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.83it/s]
2025-04-22 09:56:48,448 - sssd.utils.logger - INFO - Iteration: 5000    Loss: 0.971707284450531
2025-04-22 09:56:49,823 - sssd.utils.logger - INFO - Current time: 2025-04-22 09:56:49
Training Job completed

https://raw.githubusercontent.com/Josh-test-lab/website-assets-repository/refs/heads/main/posts/1140422%20meeting/1745285023058.png
及時資源用量。

運行環境

  • 本機作業系統：Windows 11 24H2
    • 程式語言：Python 3.12.9
  • 計算平臺：財團法人國家實驗研究院國家高速網路與計算中心臺灣 AI 雲
    • 作業系統：Ubuntu
    • Miniconda
    • GPU：NVIDIA Tesla V100 32GB GPU
    • CUDA 12.8 driver
    • 程式語言：Python 3.10.16 for Linux

延伸學習

參見

參考資料

更新於 2025-04-22 12:52 UTC+0800   4e2a537d
Creative Commons License
 Meeting, $SSSD^{S4}$, Python
返回 | 首頁

相關文章