Following is a schematic representation of a high-end Unified Storage Device (USD) setup. Two USDs are clustered for high-availability (HA) purposes. In a real setting there would be more disk shelves, switches, servers, workstations, and clients.
In the preceding illustration, the primary data paths are in red and blue. Servers, which expect raw disk storage over Fibre Channel, are served through FCP. Clients and servers, which use file-based storage or raw storage through iSCSI, are served over the Ethernet. The illustration also shows the data paths between the USDs and their disks. USDs in a cluster communicate through cluster interconnect, which is also shown (ServerNet, troika, and InfiniBand are the current possibilities, depending on the model). Each has two Fibre Channel loops out the "back end", one for its own disks and one for the disks belonging to the partner. In ordinary operation, each serves data from its own disk. If one of the controllers fails, the other does a "takeover", and begins serving data from both sets of disks. To do this, it has to take over the Ethernet address of the down partner, and pretend to actually be that system. Therefore, a USD that has been taken over but not powered off stays in a reboot loop, waiting for the signal from the takeover partner over the cluster interconnect to resume normal operation.
It is common to use a serial port concentrator (Portmaster and Annex are two well-known brands) to manage the serial port of the USD. Using one of these, you can access the serial port from any client on your network. The serial port is typically no more useful than a Telnet session to the USD (they are effectively the same thing). It is important when the system breaks, because it then might drop down to the firmware prompt. At this point there is no operating system, so Telnet cannot be used. At such a time, the serial port is the only means of managing the system.
A block diagram of the internal workings of the system is as follows:
The clients connect physically through the network and host bus adapters. The USD uses a single data path from WAFL to disk for both block- and file-based storage. This is why it is called a Unified Storage Device.
Related Topics