1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112 | ============
# mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Sun Dec 15 22:10:22 2013
Raid Level : raid6
Array Size : 749783040 (715.05 GiB 767.78 GB)
Used Dev Size : 249927680 (238.35 GiB 255.93 GB)
Raid Devices : 5
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Wed Aug 19 13:43:56 2015
State : clean, FAILED
Active Devices : 2
Working Devices : 2
Failed Devices : 1
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : vmhost:0 (local to host vmhost)
UUID : c2be06bf:96e5b370:e07dd212:f8126f4a
Events : 5759066
Number Major Minor RaidDevice State
0 0 0 0 removed
1 0 0 1 removed
6 8 48 2 active sync /dev/sdd
3 0 0 3 removed
4 8 80 4 active sync /dev/sdf
1 8 32 - faulty spare /dev/sdc
============
# dmesg | grep sdc
[16280745.588535] sd 2:0:0:0: [sdc] Unhandled error code
[16280745.588537] sd 2:0:0:0: [sdc]
[16280745.588538] sd 2:0:0:0: [sdc] CDB:
[16280745.588544] end_request: I/O error, dev sdc, sector 249261856
[16280745.605083] end_request: I/O error, dev sdc, sector 249261856
[16280745.621368] md/raid:md0: Disk failure on sdc, disabling device.
[16280745.621375] sd 2:0:0:0: [sdc] Unhandled error code
[16280745.621375] sd 2:0:0:0: [sdc]
[16280745.621376] sd 2:0:0:0: [sdc] CDB:
[16280745.621379] end_request: I/O error, dev sdc, sector 290899520
[16280745.621396] sd 2:0:0:0: [sdc] Unhandled error code
[16280745.621397] sd 2:0:0:0: [sdc]
[16280745.621398] sd 2:0:0:0: [sdc] CDB:
[16280745.621401] end_request: I/O error, dev sdc, sector 125422592
[16280745.621404] sd 2:0:0:0: [sdc] Unhandled error code
[16280745.621415] sd 2:0:0:0: [sdc]
[16280745.621416] sd 2:0:0:0: [sdc] CDB:
[16280745.621418] end_request: I/O error, dev sdc, sector 129291624
[16280745.621419] sd 2:0:0:0: [sdc] Unhandled error code
[16280745.621420] sd 2:0:0:0: [sdc]
[16280745.621420] sd 2:0:0:0: [sdc] CDB:
[16280745.621422] end_request: I/O error, dev sdc, sector 129955624
[16280745.621423] sd 2:0:0:0: [sdc] Unhandled error code
[16280745.621435] sd 2:0:0:0: [sdc]
[16280745.621435] sd 2:0:0:0: [sdc] CDB:
[16280745.621437] end_request: I/O error, dev sdc, sector 292577360
[16280745.621438] sd 2:0:0:0: [sdc] Unhandled error code
[16280745.621439] sd 2:0:0:0: [sdc]
[16280745.621439] sd 2:0:0:0: [sdc] CDB:
[16280745.621441] end_request: I/O error, dev sdc, sector 292922776
[16280745.621442] sd 2:0:0:0: [sdc] Unhandled error code
[16280745.621442] sd 2:0:0:0: [sdc]
[16280745.621443] sd 2:0:0:0: [sdc] CDB:
[16280745.621445] end_request: I/O error, dev sdc, sector 292922912
[16280745.763749] disk 1, o:0, dev:sdc
[16538056.236693] sd 2:0:0:0: [sdc] Unhandled error code
[16538056.236695] sd 2:0:0:0: [sdc]
[16538056.236698] sd 2:0:0:0: [sdc] CDB:
[16538056.236705] end_request: I/O error, dev sdc, sector 0
[16538056.249630] Buffer I/O error on device sdc, logical block 0
[16538056.262971] Buffer I/O error on device sdc, logical block 1
[16538056.262998] sd 2:0:0:0: [sdc] Unhandled error code
[16538056.263000] sd 2:0:0:0: [sdc]
[16538056.263002] sd 2:0:0:0: [sdc] CDB:
[16538056.263006] end_request: I/O error, dev sdc, sector 0
[16538056.263007] Buffer I/O error on device sdc, logical block 0
[16538056.263026] sd 2:0:0:0: [sdc] Unhandled error code
[16538056.263027] sd 2:0:0:0: [sdc]
[16538056.263029] sd 2:0:0:0: [sdc] CDB:
[16538056.263032] end_request: I/O error, dev sdc, sector 500118184
[16538056.263033] Buffer I/O error on device sdc, logical block 62514773
[16538056.263040] sd 2:0:0:0: [sdc] Unhandled error code
[16538056.263040] sd 2:0:0:0: [sdc]
[16538056.263041] sd 2:0:0:0: [sdc] CDB:
[16538056.263045] end_request: I/O error, dev sdc, sector 500118184
[16538056.263045] Buffer I/O error on device sdc, logical block 62514773
[16538056.263071] sd 2:0:0:0: [sdc] Unhandled error code
[16538056.263071] sd 2:0:0:0: [sdc]
[16538056.263072] sd 2:0:0:0: [sdc] CDB:
[16538056.263074] end_request: I/O error, dev sdc, sector 0
[16538056.263074] Buffer I/O error on device sdc, logical block 0
============
# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10]
md0 : active raid6 sdc[1](F) sdf[4] sdd[6]
749783040 blocks super 1.2 level 6, 512k chunk, algorithm 2 [5/2] [__U_U]
unused devices: <none>
|