Take A Lesson In DATA PROTECTION

Bringing flexibility to an aging legacy IT infrastructure is akin to shooting an arthritic knee with cortisone. For a time - sometimes as long as several months - the pain diminishes, and there are noticeable gains in mobility. But, inevitably, the stiffness returns. Eventually, the only way to sustain pain-free movement is to replace the knee - or re-architect the IT environment.

At school bus and specialty bus manufacturer Thomas Built Buses (TBB), the cortisone shot came in the form of a late 1990s application migration, managed by TBB's outsourced IT provider, EDS. Anticipating Y2K-related problems if it didn't make the move and improved application performance if it did, the EDS team decided to move many of TBB's enterprise apps from the legacy mainframe to a client/server environment. In addition to laying the groundwork for additional client/server-based apps, the transition fostered tremendous growth in TBB's desktop user base. "When Thomas Built brought in EDS in 1995, there were fewer than 40 PCs across the company," says Nick Lorenz, EDS' client delivery executive for TBB. "Today, we support more than 600 PCs in the non-mainframe environment."

But, as more users and more applications were brought on board, more servers were needed to support them. And, more servers meant more storage devices. "Every time we wanted to add an application, we had to add a server," says Chris Green, senior systems administrator for EDS at TBB. "We ended up with multiple application and file servers, each with its own direct attached disk arrays." Soon, the improved application performance sparked by the client/server rollout was being negated by the rigidity of the storage infrastructure. "A lot of our open systems apps run on Oracle databases, and the I/O [input/output] on Oracle was getting extremely slow," Green explains.

Clearly, the effects of the cortisone shot were wearing off. It was time to brace TBB's systems with a less vulnerable - ideally, an invulnerable - support structure. First came a SAN (storage area network) in the central data center at TBB's High Point, NC, headquarters. Then came another SAN - a hot, mirrored SAN in a remote disaster recovery site. The dual-SAN design now gives TBB a full replacement strategy should the primary SAN give out.

Open Your Applications On Open Systems
The mainframe system at TBB still performs some core processing functions. As TBB assembles buses on its shop floors, it relies heavily, for example, on MRP (materials resource planning) and production scheduling tools running on an AS/400. But, its Web apps, including internal company portals, and its engineering and CAD (computer-aided design) apps all run in the open systems environment. And, the ever-growing client/server side reflects the EDS team's preference for deploying off-the-shelf products. Says Lorenz, "We almost never take, as our first option for a new application, the step of writing some new code for mainframe processing."

The change to a client/server model has also enabled TBB to bring on e-business applications that connect TBB to its dealers. To enhance the supplier/dealer experience, TBB works to extend systems functionality to its dealers without requiring the dealers to upgrade equipment at their end. "We saw how easy we could include the dealer apps in the client/server environment," Lorenz says. "So, we brought in product configuration and forecasting tools that allow dealers to place, track, and make changes to orders. We couldn't have done those things on the mainframe."

The product configuration and forecasting applications are housed in a server farm running on the Citrix platform. Via dedicated T1 lines, dealers access the system to custom-configure buses. Although it is not alone in placing a heavy burden on TBB's storage systems, the product configurator generates compliance-related information that must be archived. "In the product configuration process, there are over 440,000 lines of rules we execute to ensure the bus is compatible with specific state and school district regulations," Lorenz explains. "That design data must be kept for years."

Dual SANs Drive Down Downtime
For the core of the primary SAN, the EDS team installed a Vivant 400 series unit from MTI Technology Corp. It also installed a second Vivant in a remote data center so TBB can replicate its SAN-housed data for disaster recovery purposes. According to Green, the SAN implementation was relatively painless. "We installed the Fibre Channel HBAs [host bus adapters] on each server that would be moved to the SAN, and created the LUNs [logical unit numbers] for carving up volumes on the SAN," he explains. "Then, we basically just copied the data to the servers on the SAN and switched over."

Supplementing the Vivant disk arrays is a StingRay NAS (network attached storage) box, also from MTI. The Vivant handles block-level data traffic, while the StingRay provides file-level access. The decision to attach a NAS device to the SAN stemmed from EDS' desire to give users file-level access to data on both UNIX and NT platforms. Dual-platform access to NAS-based storage also enabled EDS to reduce TBB's hardware overhead. Says Green, "We were able to consolidate most of the file servers onto one NAS box. That gave us the ability to draw from a single storage pool instead of adding larger and larger servers with direct attached arrays every time we needed more file storage space."

Because the SAN is designed for redundancy - with redundant HBAs, switches, and data paths - the data center can handle external shocks to its systems without having those systems suffer downtime. When a major ice storm knocked out power across North Carolina, TBB's data center did not regain steady, reliable power for more than 48 hours. In fact, one of the SAN switches was on a UPS (uninterruptible power supply) device that blew up. "When the switch went down, it failed over to another switch within seconds. We didn't suffer any data loss," Lorenz says.

In addition to enhancing TBB's data security, the SAN rollout has greatly improved application performance. Batch processing of transactions - a process that requires a nightly data dump from SAN-based Oracle databases to the AS/400 - has been cut by 1.5 hours. The performance of the product configurator has also increased. It used to take as much as five minutes for a dealer to open a new configuration session. With the SAN now pushing the product configurator's Oracle-based I/0, those minutes have been slashed to 38 to 40 seconds.

Meet Your Suppliers On The Web
With TBB's new application and storage infrastructure in place, EDS has already begun rolling out even more applications on the client/server side. A key initiative is the deployment of a digital supply chain solution built around software from webPlan. The webPlan suite enables TBB's suppliers to work interactively in the company's purchasing and scheduling systems. The Web-based supply chain solution, already being used by 80 of TBB's approximately 400 suppliers, reduces the company's traditional reliance on faxing purchase orders and manually entering received materials into the mainframe. It also makes inventory replenishment more timely. "We had a standard purchase order process set up with most suppliers, but they still had to wait to hear from us," says Lorenz. "The webPlan software sets up a workflow process that issues daily notifications to suppliers about which items Thomas Built needs, how many it needs, and when it needs them."

As the application infrastructure at TBB continues to grow, the EDS team is confident that the storage back end will keep pace. And, the need to scale is, indeed, looming. When EDS initially built the primary and remote SANs, it put 1.5 TB of capacity at each site. Currently, there are only 90 GB of available space. In addition to storing data from apps already in use, TBB is building a new production facility, which will drive storage requirements even higher. "Our data is growing exponentially, so we'll soon be buying more drives for the Vivant units," says Green. "We're just beginning to map our average growth and predict the growth from the new plant." Lorenz concludes, "We know we're going to have to scale up fast, and the SANs allow us to do that."