-
Notifications
You must be signed in to change notification settings - Fork 7
/
Antares-RAID-sparcLinux-HOWTO
7970 lines (6338 loc) · 381 KB
/
Antares-RAID-sparcLinux-HOWTO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Antares RAID SparcLinux Howto
Thom Coates1 , Carl Munio, Jim Ludemann
Revised 25 May 2002
--------------------------------------
1
Thomas D. Coates, Jr. PhD, c/o Neuropunk.Org, PO Box 910385, Lexington KY
40591-0385, [email protected]
Table of Contents
* Chapter 1 Preliminaries
+ 1.1 Preamble
+ 1.2 Acknowledgements and Thanks
+ 1.3 New Versions
* Chapter 2 Introduction
+ 2.1 5070 features5070 Main Features
+ 2.2 Background
o 2.2.1 RAID levelsRaid Levels
o 2.2.2 RAID linearRAID Linear
o 2.2.3 RAID 1Level 1
o 2.2.4 stripingStriping
o 2.2.5 RAID 0Level 0
o 2.2.6 RAID 2RAID 3Level 2 and 3
o 2.2.7 RAID 4Level 4
o 2.2.8 RAID 5Level 5
* Chapter 3 Installation
+ 3.1 compatibilitySBUS Controller Compatibility
+ 3.2 hardware installationHardware Installation Procedure
o 3.2.1 Serial Terminal
o 3.2.2 Hard Drive Plant
+ 3.3 5070 Onboard Configuration
o 3.3.1 Main Screen Options
o 3.3.2 [Q]uit
o 3.3.3 [R]aidSets:
o 3.3.4 [H]ostports:
o 3.3.5 [S]pares:
o 3.3.6 [M]onitor:
o 3.3.7 [G]eneral:
o 3.3.8 [P]robe
o 3.3.9 Example RAID Configuration Session
+ 3.4 Linux Configuration
o 3.4.1 Existing Linux Installation
o 3.4.2 New Linux Installation
+ 3.5 Maintenance
o 3.5.1 spares, activatingActivating a spare
o 3.5.2 re-integrating repaired driveRe-integrating a repaired
drive into the RAID (levels 3 and 5)
+ 3.6 Troubleshooting / Error Messages
o 3.6.1 Out of band temperature detected...
o 3.6.2 ... failed ... cannot have more than 1 faulty backend.
o 3.6.3 When booting I see: ... Sun disklabel: bad magic 0000 ...
unknown partition table.
+ 3.7 Bugs
+ 3.8 Frequently Asked Questions
o 3.8.1 How do I reset/erase the onboard configuration?
o 3.8.2 How can I tell if a drive in my RAID has failed?
+ 3.9 command referenceAdvanced Topics: 5070 Command Reference
o 3.9.1 autobootAUTOBOOT - script to automatically create all raid
sets and scsi monitors
o 3.9.2 AUTOFAULT - script to automatically mark a backend faulty
after a drive failure
o 3.9.3 AUTOREPAIR - script to automatically allocate a spare and
reconstruct a raid set
o 3.9.4 BIND - combine elements of the namespace
o 3.9.5 BUZZER - get the state or turn on or off the buzzer
o 3.9.6 CACHE - display information about and delete cache ranges
o 3.9.7 CACHEDUMP - Dump the contents of the write cache to
battery backed-up ram
o 3.9.8 CACHERESTORE - Load the cache with data from battery
backed-up ram
o 3.9.9 CAT - concatenate files and print on the standard output
o 3.9.10 CMP - compare the contents of 2 files
o 3.9.11 CONS - console device for Husky
o 3.9.12 DD - copy a file (disk, etc)
o 3.9.13 DEVSCMP - Compare a file's size against a given value
o 3.9.14 DFORMAT- Perform formatting functions on a backend disk
drive
o 3.9.15 DIAGS - script to run a diagnostic on a given device
o 3.9.16 DPART - edit a scsihd disk partition table
o 3.9.17 DUP - open file descriptor device
o 3.9.18 ECHO - display a line of text
o 3.9.19 ENV- environment variables file system
o 3.9.20 ENVIRON - RaidRunner Global environment variables - names
and effects
o 3.9.21 EXEC - cause arguments to be executed in place of this
shell
o 3.9.22 EXIT - exit a K9 process
o 3.9.23 EXPR - evaluation of numeric expressions
o 3.9.24 FALSE - returns the K9 false status
o 3.9.25 FIFO - bi-directional fifo buffer of fixed size
o 3.9.26 GET - select one value from list
o 3.9.27 GETIV - get the value an internal RaidRunner variable
o 3.9.28 HELP - print a list of commands and their synopses
o 3.9.29 HUSKY - shell for K9 kernel
o 3.9.30 HWCONF - print various hardware configuration details
o 3.9.31 HWMON - monitoring daemon for temperature, fans, PSUs.
o 3.9.32 INTERNALS - Internal variables used by RaidRunner to
change dynamics of running kernel
o 3.9.33 KILL - send a signal to the nominated process
o 3.9.34 LED- turn on/off LED's on RaidRunner
o 3.9.35 LFLASH- flash a led on RaidRunner
o 3.9.36 LINE - copies one line of standard input to standard
output
o 3.9.37 LLENGTH - return the number of elements in the given list
o 3.9.38 LOG - like zero with additional logging of accesses
o 3.9.39 LRANGE - extract a range of elements from the given list
o 3.9.40 LS - list the files in a directory
o 3.9.41 LSEARCH - find the a pattern in a list
o 3.9.42 LSUBSTR - replace a character in all elements of a list
o 3.9.43 MEM - memory mapped file (system)
o 3.9.44 MDEBUG - exercise and display statistics about memory
allocation
o 3.9.45 MKDIR - create directory (or directories)
o 3.9.46 MKDISKFS - script to create a disk filesystem
o 3.9.47 MKHOSTFS - script to create a host port filesystem
o 3.9.48 MKRAID - script to create a raid given a line of output
of rconf
o 3.9.49 MKRAIDFS - script to create a raid filesystem
o 3.9.50 MKSMON - script to start the scsi monitor daemon smon
o 3.9.51 MKSTARGD - script to initialize a scsi target daemon for
a given raid set
o 3.9.52 MSTARGD - monitor for stargd
o 3.9.53 NICE - Change the K9 run-queue priority of a K9 process
o 3.9.54 NULL- file to throw away output in
o 3.9.55 PARACC - display information about hardware parity
accelerator
o 3.9.56 PEDIT - Display/modify SCSI backend Mode Parameters Pages
o 3.9.57 PIPE - two way interprocess communication
o 3.9.58 PRANKS - print or set the accessible backend ranks for
the current controller
o 3.9.59 PRINTENV - print one or all GLOBAL environment variables
o 3.9.60 PS - report process status
o 3.9.61 PSCSIRES - print SCSI-2 reservation table for all or
specific monikers
o 3.9.62 PSTATUS - print the values of hardware status registers
o 3.9.63 RAIDACTION- script to gather/reset stats or stop/start a
raid set's stargd
o 3.9.64 RAID0 - raid 0 device
o 3.9.65 RAID1 - raid 1 device
o 3.9.66 RAID3 - raid 3 device
o 3.9.67 RAID4 - raid 4 device
o 3.9.68 RAID5 - raid 5 device
o 3.9.69 RAM - ram based file system
o 3.9.70 RANDIO - simulate random reads and writes
o 3.9.71 RCONF, SPOOL, HCONF, MCONF, CORRUPT-CONFIG - raid
configuration and spares management
o 3.9.72 REBOOT - exit K9 on target hardware + return to monitor
o 3.9.73 REBUILD - raid set reconstruction utility
o 3.9.74 REPAIR - script to allocate a spare to a raid set's
failed backend
o 3.9.75 REPLACE - script to restore a backend in a raid set
o 3.9.76 RM - remove the file (or files)
o 3.9.77 RMON - Power-On Diagnostics and Bootstrap
o 3.9.78 RRSTRACE - disassemble scsihpmtr monitor data
o 3.9.79 RSIZE - estimate the memory usage for a given raid set
o 3.9.80 SCN2681 - access a scn2681 (serial IO device) as console
o 3.9.81 SCSICHIPS - print various details about a controller's
scsi chips
o 3.9.82 SCSIHD - SCSI hard disk device (a SCSI initiator)
o 3.9.83 SCSIHP - SCSI target device
o 3.9.84 SET - set (or clear) an environment variable
o 3.9.85 SCSIHPMTR - turn on host port debugging
o 3.9.86 SETENV - set a GLOBAL environment variable
o 3.9.87 SDLIST - Set or display an internal list of attached disk
drives
o 3.9.88 SETIV - set an internal RaidRunner variable
o 3.9.89 SHOWBAT - display information about battery backed-up ram
o 3.9.90 SHUTDOWN - script to place the RaidRunner into a shutdown
or quiescent state
o 3.9.91 SLEEP - sleep for the given number of seconds
o 3.9.92 SMON - RaidRunner SCSI monitor daemon
o 3.9.93 SOS - pulse the buzzer to emit sos's
o 3.9.94 SPEEDTST - Generate a set number of sequential writes
then reads
o 3.9.95 SPIND - Spin up or down a disk device
o 3.9.96 SPINDLE - Modify Spindle Synchronization on a disk device
o 3.9.97 SRANKS - set the accessible backend ranks for a
controller
o 3.9.98 STARGD - daemon for SCSI-2 target
o 3.9.99 STAT - get status information on the named files (or
stdin)
o 3.9.100 STATS - Print cumulative performance statistics on a
Raid Set or Cache Range
o 3.9.101 STRING - perform a string operation on a given value
o 3.9.102 SUFFIX - Suffixes permitted on some big decimal
numbers
o 3.9.103 SYSLOG - device to send system messages for logging
o 3.9.104 SYSLOGD - initialize or access messages in the system
log area
o 3.9.105 TEST - condition evaluation command
o 3.9.106 TIME - Print the number of seconds since boot (or
reset of clock)
o 3.9.107 TRAP - intercept a signal and perform some action
o 3.9.108 TRUE - returns the K9 true status
o 3.9.109 STTY or TTY - print the user's terminal mount point
or terminfo status
o 3.9.110 UNSET - delete one or more environment variables
o 3.9.111 UNSETENV - unset (delete) a GLOBAL environment
variable
o 3.9.112 VERSION - print out the version of the RaidRunner
kernel
o 3.9.113 WAIT - wait for a process (or my children) to
terminate
o 3.9.114 WARBLE - periodically pulse the buzzer
o 3.9.115 XD- dump given file(s) in hexa-decimal to standard
out
o 3.9.116 ZAP - write zeros to a file
o 3.9.117 ZCACHE - Manipulate the zone optimization IO table of
a Raid Set's cache
o 3.9.118 ZERO - file when read yields zeros continuously
o 3.9.119 ZLABELS - Write zeros to the front and end of Raid
Sets
+ 3.10 Advanced Topics: SCSI Monitor Daemon (SMON)
+ 3.11 Further Reading
Chapter 1 Preliminaries
This document describes how to install, configure, and maintain a hardware
RAID built around the 5070 SBUS host based RAID controller by Antares
Microsystems. Other topics of discussion include RAID levels, the 5070
controller GUI, and 5070 command line. A complete command reference for the
5070's K9 kernel and Bourne-like shell is included.
1.1 Preamble
Copyright 2000 by Thomas D. Coates, Jr. This document's source is licensed
under the terms if the GNU general public license agreement.
Permission to use, copy, modify, and distribute this document without fee for
any purpose commercial or non-commercial is hereby granted, provided that the
author's names and this notice appear in all copies and/or supporting
documents; and that the location where a freely available unmodified version
of this document may be obtained is given. This document is distributed in
the hope that it will be useful, but WITHOUT ANY WARRANTY, either expressed
or implied. While every effort has been taken to ensure the accuracy of the
information documented herein, the author(s)/editor(s)/maintainer(s)/
contributor(s) assumes NO RESPONSIBILITY for any errors, or for any damages,
direct or consequential, as a result of the use of the information documented
herein. A complete copy of the GNU Public License agreement may be obtained
from: Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA
02111-1307, USA. Portions of this document are adapted and/or re-printed from
the 5070 installation guide and man pages with permission of Antares
Microsystems, Inc., Campbell CA.
1.2 Acknowledgements and Thanks
* Carl and Jim at Antares for the hardware, man pages, and other support/
contributions they provided during the writing of this document.
* Penn State University - Hershey Medical Center, Department of Radiology,
Section of Clinical Image Management (My home away from my home away from
home).
* The software-raid-HOWTO Copyright 1997 by Linas Vepstas under the GNU
public license agreement. The software-raid-HOWTO is Available from :
http://www.linuxdoc.org
1.3 New Versions
* The location of the most recent version of this document is posted on my
homepage: http://members.iglou.com/tcoates/
* Other versions may be found in different formats at the LDP homepage:
http://www.linuxdoc.org and mirror sites.
Chapter 2 Introduction
The Antares 5070 is a high performance, versatile, yet relatively inexpensive
host based RAID controller. Its embedded operating system (K9 kernel) is
modelled on the Plan 9 operating system whose design is discussed in several
papers from AT&T (see the "Further Reading" section). K9 is a kernel targeted
at embedded controllers of small to medium complexity (e.g. ISDN-ethernet
bridges, RAID controllers, etc). It supports multiple lightweight processes
(i.e. without memory management) on a single CPU with a non-preemptive
scheduler. Device driver architecture is based on Plan 9 (and Unix SVR4)
streams. Concurrency control mechanisms include semaphores and signals.
The 5070 has three single ended ultra 1 SCSI channels and two onboard serial
interfaces one of which provides command line access via a connected serial
terminal or modem. The other is used to upgrade the firmware. The command
line is robust, implementing many of the essential Unix commands (e.g. dd,
ls, cat, etc.) and a scaled down Bourne shell for scripting. The Unix command
set is augmented with RAID specific configuration commands and scripts. In
addition to the command line interface an ASCII text based GUI is provided to
permit easy configuration of level 0, 1, 3, 4, and 5 RAIDs.
2.1 5070 features5070 Main Features
* RAID levels 0, 1, 3, 4, and 5 are supported.
* Text based GUI for easy configuration for all supported RAID levels.
* A Multidisk RAID volume appears as an individual SCSI drive to the
operating system and can be managed with the standard utilities (fdisk,
mkfs, fsck,etc.). RAID Volumes may be assigned to different SCSI IDs or
the same SCSI IDs but different LUNs.
* No special RAID drivers required for the host operating system.
* Multiple RAID volumes of different levels can be mixed among the drives
forming the physical plant. For example in a hypothetical drive plant
consisting of 9 drives:
+ 2 drives form a level 3 RAID assigned to SCSI ID 5, LUN 0
+ 2 drives form a level 0 RAID assigned to SCSI ID 5, LUN 1
+ 5 drives form a level 5 RAID assigned to SCSI ID 6, LUN 0
* Three single ended SCSI channels which can accommodate 6 drives each (18
drives total).
* Two serial interfaces. The first permits configuration/control/monitoring
of the RAID from a local serial terminal. The second serial port is used
to upload new programming into the 5070 (using PPP and TFTP).
* Robust Unix-like command line and NVRAM based file system.
* Configurable ASCII SCSI communication channel for passing commands to the
5070's command line interpreter. Allows programming running on host OS to
directly configure/control/monitor all parameters of the 5070.
2.2 Background
Much of the information/knowledge pertaining to RAID levels in this section
is adapted from the software-raid-HOWTO by Linas Vepstas . See the
acknowledgements section for the URL where the full document may be obtained.
RAID is an acronym for "Redundant Array of Inexpensive Disks" and is used to
create large, reliable disk storage systems out of individual hard disk
drives. There are two basic ways of implementing a RAID, software or
hardware. The main advantage of a software RAID is low cost. However, since
the OS of the host system must manage the RAID directly there is a
substantial penalty in performance. Furthermore if the RAID is also the boot
device, a drive failure could prove disastrous since the operating system and
utility software needed to perform the recovery is located on the RAID. The
primary advantages of hardware RAID is performance and improved reliability.
Since all RAID operations are handled by a dedicated CPU on the controller,
the host system's CPU is never bothered with RAID related tasks. In fact the
host OS is completely oblivious to the fact that its SCSI drives are really
virtual RAID drives. When a drive fails on the 5070 it can be replaced
on-the-fly with a drive from the spares pool and its data reconstructed
without the host's OS ever knowing anything has happened.
2.2.1 RAID levelsRaid Levels
The different RAID levels have different performance, redundancy, storage
capacity, reliability and cost characteristics. Most, but not all levels of
RAID offer redundancy against drive failure. There are many different levels
of RAID which have been defined by various vendors and researchers. The
following describes the first 7 RAID levels in the context of the Antares
5070 hardware RAID implementation.
2.2.2 RAID linearRAID Linear
RAID-linear is a simple concatenation of drives to create a larger virtual
drive. It is handy if you have a number small drives, and wish to create a
single, large drive. This concatenation offers no redundancy, and in fact
decreases the overall reliability: if any one drive fails, the combined drive
will fail.
SUMMARY
* Enables construction of a large virtual drive from a number of smaller
drives
* No protection, less reliable than a single drive
* RAID 0 is a better choice due to better I/O performance
2.2.3 RAID 1Level 1
Also referred to as "mirroring". Two (or more) drives, all of the same size,
each store an exact copy of all data, disk-block by disk-block. Mirroring
gives strong protection against drive failure: if one drive fails, there is
another with the an exact copy of the same data. Mirroring can also help
improve performance in I/O-laden systems, as read requests can be divided up
between several drives. Unfortunately, mirroring is also one of the least
efficient in terms of storage: two mirrored drives can store no more data
than a single drive.
SUMMARY
* Good read/write performance
* Inefficient use of storage space (half the total space available for
data)
* RAID 6 may be a better choice due to better I/O performance.
2.2.4 stripingStriping
Striping is the underlying concept behind all of the other RAID levels. A
stripe is a contiguous sequence of disk blocks. A stripe may be as short as a
single disk block, or may consist of thousands. The RAID drivers split up
their component drives into stripes; the different RAID levels differ in how
they organize the stripes, and what data they put in them. The interplay
between the size of the stripes, the typical size of files in the file
system, and their location on the drive is what determines the overall
performance of the RAID subsystem.
2.2.5 RAID 0Level 0
Similar to RAID-linear, except that the component drives are divided into
stripes and then interleaved. Like RAID-linear, the result is a single larger
virtual drive. Also like RAID-linear, it offers no redundancy, and therefore
decreases overall reliability: a single drive failure will knock out the
whole thing. However, the 5070 hardware RAID 0 is the fastest of any of the
schemes listed here.
SUMMARY:
* Use RAID 0 to combine smaller drives into one large virtual drive.
* Best Read/Write performance of all the schemes listed here.
* No protection from drive failure.
* ADVICE: Buy very reliable hard disk drives if you plan to use this
scheme.
2.2.6 RAID 2RAID 3Level 2 and 3
RAID-2 is seldom used anymore, and to some degree has been made obsolete by
modern hard disk technology. RAID-2 is similar to RAID-4, but stores ECC
information instead of parity. Since all modern disk drives incorporate ECC
under the covers, this offers little additional protection. RAID-2 can offer
greater data consistency if power is lost during a write; however, battery
backup and a clean shutdown can offer the same benefits. RAID-3 is similar to
RAID-4, except that it uses the smallest possible stripe size.
SUMMARY
* RAID 2 is largely obsolete
* Use RAID 3 to combine separate drives together into one large virtual
drive.
* Protection against single drive failure,
* Good read/write performance.
2.2.7 RAID 4Level 4
RAID-4 interleaves stripes like RAID-0, but it requires an additional drive
to store parity information. The parity is used to offer redundancy: if any
one of the drives fail, the data on the remaining drives can be used to
reconstruct the data that was on the failed drive. Given N data disks, and
one parity disk, the parity stripe is computed by taking one stripe from each
of the data disks, and XOR'ing them together. Thus, the storage capacity of a
an (N+1)-disk RAID-4 array is N, which is a lot better than mirroring (N+1)
drives, and is almost as good as a RAID-0 setup for large N. Note that for N=
1, where there is one data disk, and one parity disk, RAID-4 is a lot like
mirroring, in that each of the two disks is a copy of each other. However,
RAID-4 does NOT offer the read-performance of mirroring, and offers
considerably degraded write performance. In brief, this is because updating
the parity requires a read of the old parity, before the new parity can be
calculated and written out. In an environment with lots of writes, the parity
disk can become a bottleneck, as each write must access the parity disk.
SUMMARY
* Similar to RAID 0
* Protection against single drive failure.
* Poorer I/O performance than RAID 3
* Less of the combined storage space is available for data [than RAID 3]
since an additional drive is needed for parity information.
2.2.8 RAID 5Level 5
RAID-5 avoids the write-bottleneck of RAID-4 by alternately storing the
parity stripe on each of the drives. However, write performance is still not
as good as for mirroring, as the parity stripe must still be read and XOR'ed
before it is written. Read performance is also not as good as it is for
mirroring, as, after all, there is only one copy of the data, not two or
more. RAID-5's principle advantage over mirroring is that it offers
redundancy and protection against single-drive failure, while offering far
more storage capacity when used with three or more drives.
SUMMARY
* Use RAID 5 if you need to make the best use of your available storage
space while gaining protection against single drive failure.
* Slower I/O performance than RAID 3
Chapter 3 Installation
NOTE: The installation procedure given here for the SBUS controller is
similar to that found in the manual. It has been modified so minor variations
in the SPARCLinux installation may be included.
3.1 compatibilitySBUS Controller Compatibility
The 5070 / Linux 2.2 combination was tested on SPARCstation (5, 10, & 20),
Ultra 1, and Ultra 2 Creator. The 5070 was also tested on Linux with
Symmetrical Multiprocessing (SMP) support on a dual processor Ultra 2 creator
3D with no problems. Other 5070 / Linux / hardware combinations may work as
well.
3.2 hardware installationHardware Installation Procedure
If your system is already up and running, you must halt the operating system.
GNOME:
1. From the login screen right click the "Options" button.
2. On the popup menu select System -> Halt.
3. Click "Yes" when the verification box appears
KDE:
1. From the login screen right click shutdown.
2. On the popup menu select shutdown by right clicking its radio button.
3. Click OK
XDM:
1. login as root
2. Left click on the desktop to bring up the pop-up menu
3. select "New Shell"
4. When the shell opens type "halt" at the prompt and press return
Console Login (systems without X windows):
1. Login as root
2. Type "halt"
All Systems:
Wait for the message "power down" or "system halted" before proceeding. Turn
off your SPARCstation system (Note: Your system may have turned itself off
following the power down directive), its video monitor, external disk
expansion boxes, and any other peripherals connected to the system. Be sure
to check that the green power LED on the front of the system enclosure is not
lit and that the fans inside the system are not running. Do not disconnect
the system power cord.
SPARCstation 4, 5, 10, 20 & UltraSPARC Systems:
1. Remove the top cover on the CPU enclosure. On a SPARCstation 10, this is
done by loosening the captive screw at the top right corner of the back
of the CPU enclosure, then tilting the top of the enclosure forward while
using a Phillips screwdriver to press the plastic tab on the top left
corner.
2. Decide which SBUS slot you will use. Any slot will do. Remove the filler
panel for that slot by removing the two screws and rectangular washers
that hold it in.
3. Remove the SBUS retainer (commonly called the handle) by pressing outward
on one leg of the retainer while pulling it out of the hole in the
printed circuit board.
4. Insert the board into the SBUS slot you have chosen. To insert the board,
first engage the top of the 5070 RAIDium backpanel into the backpanel of
the CPU enclosure, then rotate the board into a level position and mate
the SBUS connectors. Make sure that the SBUS connectors are completely
engaged.
5. Snap the nylon board retainers inside the SPARCstation over the 5070
RAIDium board to secure it inside the system.
6. Secure the 5070 RAIDium SBUS backpanel to the system by replacing the
rectangular washers and screws that held the original filler panel in
place.
7. Replace the top cover by first mating the plastic hooks on the front of
the cover to the chassis, then rotating the cover down over the unit
until the plastic tab in back snaps into place. Tighten the captive screw
on the upper right corner.
Ultra Enterprise Servers, SPARCserver 1000 & 2000 Systems, SPARCserver
6XO MP Series:
1. Remove the two Allen screws that secure the CPU board to the card cage.
These are located at each end of the CPU board backpanel.
2. Remove the CPU board from the enclosure and place it on a static-free
surface.
3. Decide which SBUS slot you will use. Any slot will do. Remove the filler
panel for that slot by removing the two screws and rectangular washers
that hold it in. Save these screws and washers.
4. Remove the SBUS retainer (commonly called the handle) by pressing outward
on one leg of the retainer while pulling it out of the hole in the
printed circuit board.
5. Insert the board into the SBUS slot you have chosen. To insert the board,
first engage the top of the 5070 RAIDium backpanel into the backpanel of
the CPU enclosure, then rotate the board into a level position and mate
the SBUS connectors. Make sure that the SBUS connectors are completely
engaged.
6. Secure the 5070 RAIDium board to the CPU board with the nylon screws and
standoffs provided on the CPU board. The standoffs may have to be moved
so that they match the holes used by the SBUS retainer, as the standoffs
are used in different holes for an MBus module. Replace the screws and
rectangular washers that originally held the filler panel in place,
securing the 5070 RAIDium SBus backpanel to the system enclosure.
7. Re-insert the CPU board into the CPU enclosure and re-install the
Allen-head retaining screws that secure the CPU board.
All Systems:
1. Mate the external cable adapter box to the 5070 RAIDium and gently
tighten the two screws that extend through the cable adapter box.
2. Connect the three cables from your SCSI devices to the three 68-pin
SCSI-3 connectors on the Antares 5070 RAIDium. The three SCSI cables must
always be reconnected in the same order after a RAID set has been
established, so you should clearly mark the cables and disk enclosures
for future disassembly and reassembly.
3. Configure the attached SCSI devices to use SCSI target IDs other than 7,
as that is taken by the 5070 RAIDium itself. Configuring the target
number is done differently on various devices. Consult the manufacturer's
installation instructions to determine the method appropriate for your
device.
4. As you are likely to be installing multiple SCSI devices, make sure that
all SCSI buses are properly terminated. This means a terminator is
installed only at each end of each SCSI bus daisy chain.
Verifying the Hardware Installation:
These steps are optional but recommended. First, power-on your system and
interrupt the booting process by pressing the "Stop" and "a" keys (or the
"break" key if you are on a serial terminal) simultaneously as soon as the
Solaris release number is shown on the screen. This will force the system to
run the Forth Monitor in the system EPROM, which will display the "ok"
prompt. This gives you access to many useful low-level commands, including:
ok show-devs
. . .
/iommu@f,e0000000/sbus@f,e000100SUNW, isp@1,8800000
. . .
The first line in the response shown above means that the 5070 RAIDium host
adapter has been properly recognized. If you don't see a line like this, you
may have a hardware problem.
Next, to see a listing of all the SCSI devices in your system, you can use
the probe-scsi-all command, but first you must prepare your system as
follows:
ok setenv auto-boot? False
ok reset
ok probe-scsi-all
This will tell you the type, target number, and logical unit number of every
SCSI device recognized in your system. The 5070 RAIDium board will report
itself attached to an ISP controller at target 0 with two Logical Unit
Numbers (LUNs): 0 for the virtual hard disk drive, and 7 for the connection
to the Graphical User Interface (GUI). Note: the GUI communication channel on
LUN 7 is currently unused under Linux. See the discussion under "SCSI Monitor
Daemon (SMON)" in the "Advanced Topics" section for more information.
REQUIRED: Perform a reconfiguration boot of the operating system:
ok boot -r
If no image appears on your screen within a minute, you most likely have a
hardware installation problem. In this case, go back and check each step of
the installation procedure. This completes the hardware installation
procedure.
3.2.1 Serial Terminal
If you have a serial terminal at your disposal (e.g. DEC-VT420) it may be
connected to the controller's serial port using a 9 pin DIN male to DB25 male
serial cable. Otherwise you will need to supplement the above cable with a
null modem adapter to connect the RAID controller's serial port to the serial
port on either the host computer or a PC. The terminal emulators I have
successfully used include Minicom (on Linux), Kermit (on Caldera's Dr. DOS),
and Hyperterminal (on a windows CE palmtop), however, any decent terminal
emulation software should work. The basic settings are 9600 baud , no parity,
8 data bits, and 1 stop bit.
3.2.2 Hard Drive Plant
Choosing the brand and capacity of the drives that will form the hard drive
physical plant is up to you. I do have some recommendations:
* Remember, you generally get what you pay for. I strongly recommend paying
the extra money for better (i.e. more reliable) hardware especially if
you are setting up a RAID for a mission critical project. For example,
consider purchasing drive cabinets with redundant hot-swappable power
supplies, etc.
* You will also want a UPS for your host system and drive cabinets.
Remember, RAID levels 3 and 5 protect you from data loss due to drive
failure NOT power failure.
* The drive cabinet you select should have hot swappable drive bays, these
cost more but are definitely worth it when you need to add/change drives.
* Make sure the cabinet(s) have adequate cooling when fully loaded with
drives.
* Keep your SCSI cables (internal and external) as short as possible
* Mark the drives/cabinet(s) in such a way that you will be able to
reconnect them to the controller in their original configuration. Once
the RAID is configured you cannot re-organize you drives without
re-configuring the RAID (and subsequently erasing the data stored on it).
* Keep in mind that although it is physically possible to connect/configure
up to 6 drives per channel, performance will sharply decrease for RAIDs
with more than three drives per channel. This is due to the 25 MHz
bandwidth limitation of the SBUS. Therefore, if read/write performance is
an issue go with a small number of large drives. If you need a really
large RAID (~ 1 terabyte) then you will have no other choice but to load
the channels to capacity and pay the performance penalty. NOTE: if you
are serving files over a 10/100 Base T network you may not notice the
performance decrease since the network is usually the bottleneck not the
SBUS.
3.3 5070 Onboard Configuration
Before diving into the RAID configuration I need to define a few terms.
* "RaidRunner" is the name given to the the 5070 controller board.
* "Husky" is the name given to the shell which produces the ":raid;"
command prompt. It is a command language interpreter that executes
commands read from the standard input or from a file. Husky is a scaled
down model of Unix's Bourne shell (sh). One major difference is that
husky has no concept of current working directory. For more information
on the husky shell and command prompt see the "Advanced Topics" section
* The "host port" is the SCSI ID assigned to the controller card itself.
This is usually ID 7.
* A "backend" is a drive attached to the controller on a given channel.
* A "rank" is a collection of all the backends from each channel with the
same SCSI ID
(i.e. rank 0 would consist of all the drives with SCSI ID 0 on each
channel)
* Each of the backends is identified by a three digit number where the
first digit is the channel, the second the SCSI ID of the drive, and the
third the LUN of the drive. The numbers are separated by a period. The
identifier is prefixed with a "D" if it is a disk or "T" if it is a tape
(e.g. D0.1.0). This scheme is referred to as <device_type_c.s.l> in the
following documentation.
* A "RAID set" consists of given number of backends (there are certain
requirements which I'll come to later)
* A "spare" is a drive which is unused until there is a failure in one of
the RAID drives. At that time the damaged drive is automatically taken
offline and replaced with the spare. The data is then reconstructed on
the spare and the RAID resumes normal operation.
* Spares may either be "hot" or "warm" depending on user configuration. Hot
spares are spun up when the RAID is started, which shortens the
replacement time when a drive failure occurs. Warm spares are spun up
when needed, which saves wear on the drive.
The test based GUI can be started by typing "agui"
: raid; agui
at the husky prompt on the serial terminal (or emulator).
Agui is a simple ASCII based GUI that can be run on the RaidRunner console
port which enables one to configure the RaidRunner. The only argument agui
takes is the terminal type that is connected to the RaidRunner console.
Current supported terminals are dtterm, vt100 and xterm. The default is
dtterm.
Each agui screen is split into two areas, data and menu. The data area, which
generally uses all but the last line of the screen, displays the details of
the information under consideration. The menu area, which generally is the
bottom line of the screen, displays a strip menu with a title then list of
options or sub-menus. Each option has one character enclosed in square
brackets (e.g. [Q]uit) which is the character to type to select that option.
Each menu line allows you to refresh the screen data (in case another process
on the RaidRunner writes to the console). The refresh character may also be
used during data entry if the screen is overwritten. The refresh character is
either <Control-l> or <Control-r>.
When agui starts, it reads the configuration of the RaidRunner and probes for
every possible backend. As it probes for each backend, it's "name" is
displayed in the bottom left corner of the screen.
3.3.1 Main Screen Options
The Main screen (Figure 3.1) is the first screen displayed. It provides a
summary of the RaidRunner configuration. At the top is the RaidRunner model,
version and serial number. Next is a line displaying, for each controller,
the SCSI ID's for each host port (labeled A, B, C, etc) and total and
currently available amounts of memory. The next set of lines display the
ranks of devices on the RaidRunner. Each device follows the nomenclature of <
device_type_c.s.l> where device_type_ can be D for disk or T for tape, c is
the internal channel the device is attached to, s is the SCSI ID (Rank) of
the device on that channel, and l is the SCSI LUN of the device (typically
0).
---------------------------------------------------------
[antares-RAID-SparcLinux-HOWTO001]
Figure 3.1: The main screen of the 5070 onboard configuration utility
---------------------------------------------------------
The next set of lines provide a summary of the Raid Sets configured on the
RaidRunner. The summary includes the raid set name, it's type, it's size, the
amount of cache allocated to it and a comma separated list of it's backends.
See rconf in the "Advanced Topics" section for a full description of the
above.
Next the spare devices are configured. Each spare is named (device_type_c.s.l
format), followed by it's size (in 512-byte blocks), it's spin state (Hot or
Warm), it's controller allocation , and finally it's current status (Used/
Unused, Faulty/Working). If used, the raid set that uses it is nominated.
At the bottom of the data area, the number of controllers, channels, ranks
and devices are displayed.
The menu line allows one to quit agui or select further actions or sub-menus.
* [Q]uit: Exit the main screen and return to the husky prompt.
* [R]aidSets: Enter the RaidSet configuration screen.
* [H]ostports Enter the Host Port configuration screen.
* [S]pares Enter the Spare Device configuration screen.
* [M]onitor Enter the SCSI Monitor configuration screen.
* [G]eneral Enter the General configuration/information screen.
* [P]robe Re-probe the device backends on the RaidRunner. As each backend
is probed it's "name" (c.s.l format) is displayed in the bottom left
corner of the screen.
These selections are described in detail below.
3.3.2 [Q]uit
Exit the agui main screen and return to the husky ( :raid; ) prompt.
3.3.3 [R]aidSets:
The Raid Set Configuration screen (Figure 3.2) displays a Raid Set in the
data area and provides a menu which allows you to Add, Delete, Modify,
Install (changes) and Scroll through all other raid sets (First, Last, Next
and Previous). If no raid sets have been configured, only the screen title
and menu is displayed. All attributes of the raid set are displayed. For
information on each attribute of the raid set, see the rconf command in the
"Advanced Topics" section. The menu line allows one to leave the Raid Set
Configuration screen or select further actions.
---------------------------------------------------------
[antares-RAID-SparcLinux-HOWTO002]
Figure 3.2: The RAIDSet configuration screen.
---------------------------------------------------------
* [Q]uit: Exit the Raid Set Configuration screen and return to the Main
screen. If you have modified, deleted or added a raid set and have not
installed the changes you will be asked to confirm this. If you select
Yes to continue the exit, all changes made since the last install action
will be discarded.
* [I]nst: This action installs (into the RaidRunner configuration area) any
changes that may have been made to raid sets, be that deletion, addition
or modification. If you exit prior to installing, all changes made since
the last installation will be discarded. The installation process takes
time. It is complete once the typed "i" character, is cleared from the
menu line.
* [M]od: This action allows you to modify the displayed raid set. You will
be prompted for each Raid Set attribute that can be changed. The prompt
includes allowable options or formats required. If you don't wish to
change a particular attribute, then press the RETURN or TAB key. The
attributes you can change are the raid set name, I/O mode, status (Active
to Inactive), bootmode, spares usage, backend zone table usage, IO size
(if raid set has never been used - i.e. just added), cache size, I/O
queues length, host interfaces and additional stargd arguments. If you
wish to change a single attribute then use the RETURN or TAB key to skip
all other options. The changed attribute will be re-displayed as soon as
you press the RETURN key. When specifying cache size, you may suffix the
number with 'm' or 'M' to indicate the number is in Megabytes or with 'k'
or 'K' to indicate the number is in Kilobytes. Note you can only enter
whole integer values. When specifying io size, you may suffix the number
with 'k' or 'K' to indicate the number is in Kilobytes. When you enter
data, it is checked for correctness and if incorrect, a message is
displayed and all changes are discarded and you will have to start again.
Remember you must install ([I]nst.) any changes.
* [A]dd: When this option is selected you will be prompted for various
attributes of the new raid set. These attributes are the raid set name,
the raid set type, the initial host interface the raid set is to appear
on (in c.h.l format where c is the controller number, h is the host port
(0, 1, 2 etc) and l is the SCSI LUN) and finally a list of backends. When
backends are to be entered, the screen displays a list of available
backends, each with a numeric index (commencing at 0). You select each
backend by entering the index and once complete enter q for Quit. As each
backend index is entered, it's backend name is displayed in a comma
separated list. When you enter data, it is checked for correctness and if
incorrect, a message is displayed and the addition will be ignored and
you will have to start again. Once the backends are complete, the newly
created raid set will be displayed on the screen with supplied and
default attributes. You can then modify the raid set to change other
attributes. Remember you must install ([I]nst.) any new raid sets.
* [D]elete: This action will delete the currently displayed raid set. If
this raid set is Active, then you will not be allowed to delete it. You
will have to make it Inactive (via the [M]od. option) then delete it. You
will be prompted to confirm the deletion. Once you confirm the deletion,
the screen will be cleared and the next raid set will be displayed, if
configured. Remember you must install ([I]nst.) any changes.
* [F]irst, [L]ast, [N]ext and [P]rev allow you to scroll through the
configured raid sets.
3.3.4 [H]ostports:
The Host Port Configuration screen (Figure 3.3) displays for each controller,
each host port (labelled A, B, C, etc for port number 0, 1, 2, etc) and the
assigned SCSI ID. If the RaidRunner you use, has external switches for host
port SCSI ID selection, you may only exit ([Q]uit) from this screen. If the
RaidRunner you use, does NOT have external switches for host port SCSI ID
selection, then you may modify (and hence install) the SCSI ID for any host
port. The menu line allows one to leave the Host Port Configuration screen or
select further actions (if NO external host).
---------------------------------------------------------
[antares-RAID-SparcLinux-HOWTO003]
Figure 3.3: The host port configuration screen.
---------------------------------------------------------
* [Q]uit: Exit the Host Port Configuration screen and return to the Main
screen. If you have modified a host port SCSI ID assignment and have not
installed the changes you will be asked to confirm this. If you select
Yes to continue the exit, all changes made since the last install action
will be discarded.
* [I]nstall: This action installs (into the RaidRunner configuration area)
any changes that may have been made to host port SCSI ID assign ments.
If you exit prior to installing, all changes made since the last
installation will be discarded. The installation process takes time. It
is complete once the typed "i" character, is cleared from the menu line.
* [M]odify: This action allows you to modify the host port SCSI ID
assignments for each host port on each controller (if NO external host
port SCSI ID switches). You will be prompted for the SCSI ID for each
host port. You can enter either a SCSI ID (0 thru 15), the minus "-"
character to clear the SCSI ID assignment or RETURN to SKIP. As you enter
data, it is checked for correctness and if incorrect, a message will be
printed although previously correctly entered data will be retained.
Remember you must install ([I]nst.) any changes.
3.3.5 [S]pares:
The Spare Device Configuration screen (Figure 3.4) displays all configured
spare devices in the data area and provides a menu which allows you to Add,
Delete, Mod ify and Install (changes) spare devices. If no spare devices
have been configured, only the screen title and menu is displayed. Each spare
device displayed, shows it's name (in device_type_c.s.l format), it's size in
512-byte blocks, it's spin status (Hot or Warm), it's controller allocation,
finally it's current status (Used/Unused, Faulty/Working). If used, the raid
set that uses it is nominated. For information on each attribute of a spare
device, see the rconf command in the "Advanced Topics" section. The menu line
allows one to leave the Spare Device Configuration screen or select further
actions.
---------------------------------------------------------
[antares-RAID-SparcLinux-HOWTO004]
Figure 3.4: The spare device configuration screen.
---------------------------------------------------------
* [Q]uit: Exit the Spare Device Configuration screen and return to the Main
screen. If you have modified, deleted or added a spare device and have
not installed the changes you will be asked to confirm this. If you
select Yes to continue the exit, all changes made since the last install
action will be discarded.
* [I]nstall: This action installs (into the RaidRunner configuration area)
any changes that may have been made to the spare devices, be that
deletion, addition or modification. If you exit prior to installing, all
changes made since the last installation will be discarded. The
installation process takes time. It is complete once the typed "i"
character, is cleared from the menu line.
* [M]odify: This action allows you to modify the unused spare devices. You
will be prompted for each spare device attribute that can be changed. The
prompt includes allowable options or formats required. If you don't wish
to change a particular attribute, then press the RETURN key. The
attributes you can change are the new size (in 512-byte blocks), the spin
state (H or hot or W for Warm), and the controller allocation (A for any,
0 for controller 0, 1 for controller 1, etc). If you wish to change a
single attribute of a spare device, then use the RETURN key to skip all
other attributes for each spare device. The changed attribute will not be
re-displayed until the last prompted attribute is entered (or skipped).
When you enter data, it is checked for cor rectness and if incorrect, a
message is dis played and all changes are discarded and you will have to
start again. Remember you must install ([I]nstall) any changes.
* [A]dd: When adding a spare device, the list of available devices is
displayed and you are required to type in the device name. Once entered,
the spare is added with defaults which you can change, if required, via
the [M]odify option. Remember you must install ([I]nstall) any changes.
* [D]elete: When deleting a spare device, the list of spare devices allowed
to be deleted is displayed and you are required to type in the required
device name. Once entered, the spare is deleted from the screen. Remember
you must install ([I]nstall) any changes.
3.3.6 [M]onitor:
The SCSI Monitor Configuration screen (Figure 3.5) displays a table of SCSI
monitors configured for the RaidRunner. Up to four SCSI monitors may be
configured. The table columns are entitled Controller, Host Port, SCSI LUN
and Protocol and each line of the table shows the appropriate SCSI Monitor
attribute. For details on SCSI Monitor attributes, see the rconf command in
the "Advanced Topics" section. The menu line allows one to leave the SCSI
Monitor Configuration screen or modify and install the table.
---------------------------------------------------------
[antares-RAID-SparcLinux-HOWTO005]
Figure 3.5: The SCSI monitor configuration screen.
---------------------------------------------------------
* [Q]uit: Exit the SCSI Monitor Configuration screen and return to the Main
screen. If you have made changes and have not installed them you will be
asked to confirm this. If you select Yes to continue the exit, all
changes made since the last install action will be discarded.
* [I]nstall: This action installs (into the RaidRunner configuration area)
any changes that may have been made to SCSI Monitor configuration. If you
exit prior to installing, all changes made since the last installation
will be discarded. The installation process takes time. It is complete
once the typed "i" character, is cleared from the menu line.
* [M]odify: This action allows you to modify the SCSI Monitor
configuration. The cursor will be moved around the table, prompting you
for input. If you do not want to change an attribute, enter RETURN to
skip. If you want to delete a SCSI monitor then enter the minus "-"
character when prompted for the controller number. If you want to use the
default protocol list, then enter RETURN at the Protocol List prompt. As
you enter data, it is checked for correctness and if incorrect, a message
will be printed and any previously entered data is discarded. You will