Extremely Robust, Massive Data Store

Each Hour, all Pegasus RDS™ data is finalized, and permanently stored in the RDS (until then it is fully available, but considered provisional). Permanently stored data is contained in specialized binary files (called HSBs) designed to be recoverable even in the extremely unlikely event of partial file corruption. Note that all Pegasus Data Servers use Raid 10 Disk Arrays, making file loss, even on a single Pegasus Data Server, due to hardware failures very unlikely.

The Pegasus RDS™ uses 2 completely different Binary Data File storage techniques, one optimized for writing real-time data, the other optimized for reading Historical Real-time Data. Once data for an Hour has been finalized into the 2nd Binary Data File format it is never again modified, in any way, in the RDS, further preventing the accidental corruption of already stored data. The read-only nature of the RDS is critical to satisfying regulatory requirements for permanent record maintenance.

To support an extremely large, high-performance RDS with several years of online storage, 2 completely different, highly complementary data compression techniques are used for all Binary Data Files (called HSBs). These data compression techniques also improve the RDS’ I/O Performance, as highly compressed data results in far fewer disk head moves (the single largest determinant of how fast Historical Data can be read from disk).

All access to the RDS is strictly read-only by the Pegasus Data APIs (used by all Clients); further reducing the chance that data already stored can be accidentally corrupted. The Data Server Processes (resident on each Pegasus Data Server) that serve near real-time and historical data both perform extensive data error checking as well as extensive logging for each Client. Data Server and Client logs are maintained in rolling 30 day buffers, to assist in troubleshooting. Each Data Server Process is fully multi-threaded to take full advantage of Multi-Core CPUs, and to insure that a single Client cannot block access to the RDS by other Clients.

All RDS data is constantly monitored by a set of processes which record statistics about the state and contents of the RDS to multiple Pegasus Server Databases. This information is updated in real-time and can be easily accessed by System Management personnel to monitor the health of the Pegasus RDS™.