By John Wemekamp
As battlefield communications become increasingly distributed and network-centric, a sea change has begun in the design of embedded computing systems for defense and aerospace applications. Driven by the rapidly increasing demand for distributed switching and large quantities of streaming I/O, system designers are looking to serial switch fabric architectures to provide an interconnect architecture able to deliver cost-effective systems with bandwidth and performance levels unreachable by traditional VME-based bused systems.
The good news is the availability of standards-based serial-switch-fabric implementations within existing VME systems, which designers are implementing in demanding applications such as digital signal processing (DSP). These DSP designers, typically the first to adopt high-performance architectures, need serial-switch fabrics such as StarFabric to move data quickly among the DSP board’s high-speed processors. StarFabric is an ideal platform for integrating serial-switch fabric in a 6U VME chassis.
Meanwhile, industry experts are making great progress toward defining the next generation of high-speed serial-switched-fabric VME systems, based on the new VITA 46 standard.
Today’s military and aerospace applications demand a large quantity of high-speed serial backplane I/O connectivity for sensors and storage. The demand for these high-bandwidth embedded systems is clear - over the last 18 months the commercial off-the-shelf (COTS) industry has witnessed a growing trend whereby designers supplement VMEbus parallel protocols with co-resident serial switch fabrics. Ultimately, these high-performance applications will be served with COTS systems based entirely on serial switch fabrics. Although the standards for such systems are currently in development, designers need not wait. A new generation of high-speed serial switched interconnect technologies is now available to serve as the secondary “backplane” in VME systems.
Switch Fabrics on VME
On 6U VME boards available today, designers at Curtiss-Wright Controls Embedded Computing in Leesburg, Va., use the StarFabric technology from StarGen Inc. in Marlborough, Mass., to perform the critical interconnection function. A StarFabric link operates at 2.5 gigabits per second in each direction (4 LVDS pairs at 622 megabits per second). After accounting for clock encoding and transport layer overheads, the real data throughput on a single link is approximately 220 megabytes per second.
A StarFabric link is full duplex, so the total capacity on the link is 440 megabytes per second. Curtiss-Wright has demonstrated 1.05-gigabyte-per-second full-duplex (2.1 gigabytes per second total) throughput between two DSP cards, a feat unmatched on a VME card over backplane connections. By incorporating local StarFabric switches in addition to StarFabric-to-PCI bridges, a multiprocessor board can accommodate as many as eight StarFabric ports, supporting traffic originating and terminating at the card, as well as traffic that routes between other cards in the system. This distributed-switching concept helps designers craft systems without active backplanes or space-consuming switch slots a key feature for military embedded systems.
StarFabric has other attributes of practical interest. The first is the concept of bundled links. StarFabric can take advantage of two parallel links between nodes by dynamically striping data across them, creating a virtual link with twice the bandwidth. This is transparent at the software layer, and is simpler than discretely managing two independent connections to achieve a similar result.
StarFabric is founded upon a credit-based flow-control system; receiving nodes grant credits to transmitting nodes only when they can accept a packet. Data moves when the source has sufficient credits that permit it to transmit to the next node in the fabric. The system is nonblocking because edge-node congestion does not impact traffic flow to other edge nodes.
For the real-time systems designer, StarFabric supports a class-of-service capability; credits are allocated separately for different classes of service. StarFabric provides eight classes of service, which permit a user to rank packet flow within the fabric. This capability ensures that deadline-critical data transmissions reach their destination on time.
A subtler benefit to the real-time developer is the ability to mix real-time and non-real-time data safely on the same fabric. A developer who has achieved a correctly functioning real-time system has the option of further exploiting the remaining unused bandwidth in that system for noncritical data, without fear of disrupting the system real-time design.
Software Speeds Design
To ensure interoperability across platforms, Curtiss-Wright developed the InterProcessor Communication Library (IPC), an operating-system-independent library of functions designed to enable high-performance, low-latency message passing.
IPC enables processors to communicate task-to-task on the same card or within the StarFabric network. It also provides low-overhead block data transfers, segmented block data transfers, and signalling between processors to assist in high-bandwidth data movement. Interprocessor communications provides a lightweight, high-performance messaging and data-transport API, will ensure the delivery of messages in order, and supports multiple priority levels.
Messaging occurs between tasks, which can be resident on the same processor, on the same board, or on multiple boards connected via StarFabric. The software was designed with dynamic name-space management to determine the location and routing for source and destination at runtime, as applications open endpoints for input or output. This is accomplished without the use of a central database, thus eliminating single points of failure because routing information is stored within each node.
Because sending and receiving tasks identify themselves with a name, hardware dependency is not built into the application software level, making the job of scaling systems to meet a performance goal much easier. Tasks can be moved from one processor to another without any software changes. IPC is designed to be hardware- and operating-system-neutral, to support portability to future generations of switched interconnect technologies. It will also support future interconnect fabrics such as Advanced Switching Interconnect and/or RapidIO.
Over the Horizon: VITA 46
Although serial-switch fabrics can deploy in today’s VME systems, members of the VME vendor community are working together to develop the next generation of 6U serial-switch-fabric boards and systems: VITA 46, which will provide functionally dense system architectures and a completely redesigned mechanical layout for the backplane.
The proposed standard calls for complete replacement of the DIN connectors with multiple seven-row, Multi-Gig RT2 connectors. The total backplane combination that retains VMEbus electrical connectivity and supports four x4 ports of serial-switch-fabric connectivity (sRIO, PCIe, or ASI) will then offer 264 user I/O signal pins, of which 128 can be differential pairs. This contrasts to the current VME64x backplane connectivity of 205 single-ended user I/O pins on a five-row VME module.
This new architecture features a scaleable high-speed fabric designed for distributed switching and high-speed I/O. It also addresses the previous lack of what some would describe as a truly useable 3U form factor, which VME has needed for decades. Comparatively, an alternative 6U VME variation, VITA 41, has only 30 differential pairs and has 95 fewer I/O pins than VME64x, meaning the VITA 41 user can get only two x4 ports out the back for a centralized switching approach. Our customers want distributed switching and lots of streaming I/O, while still maintaining the low-speed discrete I/O.
VME’s historical advantage is that it accommodates technological change quickly and easily, which minimizes the design effort so that customers are free to concentrate on what they do best - their end-application software. To maintain maximum interoperability and backward compatibility, attributes which have proven to be the hallmarks of the VMEbus architecture and have helped to make it the most successful embedded standard for more than 20 years, a hybrid backplane has been developed that enables both legacy and VITA 46 architectures to coexist in a single chassis. Deploying serial-switch fabrics such as StarFabric on today’s VME boards provides an immediate solution for system designers who need maximum bandwidth now. VITA 46 promises a bridge to future levels of performance that can’t be reached with today’s hardware, while providing a path that protects a customer’s investment.
John Wemekampis chief technology officer of Curtiss-Wright Controls Embedded Computing in Leesburg, Va.
