IN32A-03
UNIDATA LDM-7: A HYBRID MULTICAST/UNICAST SYSTEM FOR HIGHLY EFFICIENT AND RELIABLE REAL-TIME DATA DISTRIBUTION
Wednesday, 16 December 2015: 10:50
2020 (Moscone West)
Steven R Emmerson1, Malathi Veeraraghavan2, Shawn Chen3 and Xiang Ji3, (1)University Corporation for Atmospheric Research, Boulder, CO, United States, (2)University of Virginia Main Campus, Charles L. Brown Department of Electrical & Computer Engineering, Charlottesville, VA, United States, (3)University of Virginia Main Campus, Computer Engineering, School of Engineering & Applied Sciences, Charlottesville, VA, United States
Abstract:
Results of a pilot deployment of a major new version of the Unidata Local Data Manager (LDM-7) are presented. The Unidata LDM was developed by the University Corporation for Atmospheric Research (UCAR) and comprises a suite of software for the distribution and local processing of data in near real-time. It is widely used in the geoscience community to distribute observational data and model output, most notably as the foundation of the Unidata Internet Data Distribution (IDD) system run by UCAR, but also in private networks operated by NOAA, NASA, USGS, etc. The current version, LDM-6, uses at least one unicast TCP connection per receiving host. With over 900 connections, the bit-rate of total outgoing IDD traffic from UCAR averages approximately 3.0 GHz, with peak data rates exceeding 6.6 GHz. Expected increases in data volume suggest that a more efficient distribution mechanism will be required in the near future. LDM-7 greatly reduces the outgoing bandwidth requirement by incorporating a recently-developed “semi-reliable” IP multicast protocol while retaining the unicast TCP mechanism for reliability. During the summer of 2015, UCAR and the University of Virginia conducted a pilot deployment of the Unidata LDM-7 among U.S. university participants with access to the Internet2 network. Results of this pilot program, along with comparisons to the existing Unidata LDM-6 system, are presented.
