skip to main content
10.1145/1065983.1066022acmconferencesArticle/Chapter ViewAbstractPublication PagesmmsysConference Proceedingsconference-collections
Article

Platform Overlays: enabling in-network stream processing in large-scale distributed applications

Published: 13 June 2005 Publication History

Abstract

The purpose of this research is to explore the capabilities of future, multi-core heterogeneous systems, with specialized communication support, to be used as efficient and flexible execution platforms in distributed streaming applications. On such platforms, we create overlays of hardware- and software-supported execution contexts -- platform overlays. Stream manipulations, represented via stream handlers, are deployed on top of such overlays, based on the ability of individual contexts to perform handler operations. As a result, stream processing is dynamically mapped to those platform resources best suited for it, and it can even be fully contained to the networking subsystems, thereby enabling in-network stream processing. Experimental results demonstrate the benefits of our approach towards meeting application-specific quality requirements.

References

[1]
G. Apostolopoulos, D. Aubespin, V. Peris, P. Pradhan, and D. Saha. Design, Implementation and Performance of a Content-Based Switch. In Proc. of INFOCOM 2000.
[2]
F. Braun, J. Lockwood, and M. Waldvogel. Protocol wrappers for layered network packet processing in reconfigurable networks. IEEE Micro, Jan./Feb. 2002.
[3]
Cell Processor Architecture Explained. http://www.blachford.info/computer/Cells/Cell0.html.
[4]
C. Consel, H. Hamdi, L. Reveillere, lenin Singaravelu, H. Yu, and C. Pu. Spidle: A DSL Approach to Specifying Streaming Applications. In Proceedings of the 2nd International Conference on Generative Programming and Component Engineering, Erfurt, Germany, Sept. 2003.
[5]
A. Gavrilovska, K. Schwan, O. Nordstrom, and H. Seifu. Network Processors as Building Blocks in Overlay Networks. In Proc. of HotI 11, Stanford, CA, Aug. 2003.
[6]
E. Kohler, R. Morris, B. Chen, J. Jannotti, and M. F. Kaashoek. The Click Modular Router. ACM Transactions on Computer Systems, 18(3):263--297, Aug. 2000.
[7]
J. Kong and K. Schwan. KStreams: Kernel Support for Efficient Data Streaming in Proxy Servers. In Proc. of NOSSDAV'05, Skamania, WA, 2005.
[8]
J. Liu, J. Wu, S. P. Kini, P. Wyckoff, and D. K. Panda. High Performance RDMA-Based MPI Implementation over InfiniBand. In Int'l Conference on Supercomputing, 2003.
[9]
V. Oleson, K. Schwan, G. Eisenhauer, B. Plale, C. Pu, and D. Amin. Operational Information Systems - An Example from the Airline Industry. In First Workshop on Industrial Experiences with Systems Software (WIESS), Oct. 2000.
[10]
B. Raman and R. Katz. An Architecture for Highly Available Wide-Area Service Composition. Computer Communications Journal, May 2003.
[11]
G. Regnier, D. Minturn, G. McAlpine, V. Saletore, and A. Foong. ETA: Experience with an Intel Xeon Processor as a Packet Processing Engine. In Proc. of HotI 11, 2003.
[12]
S. Roy, J. Ankcorn, and S. Wee. An Architecture for Componentized, Network-Based Media Services. In Proc. of IEEE International Conference on Multimedia and Expo, July 2003.
[13]
P. Shivam, P. Wyckoff, and D. Panda. Can User Level Protocols Take Advantage of Multi-CPU NICs? In Int'l Parallel and Distributed Processing Symposium, 2002.
[14]
T. Spalink, S. Karlin, L. Peterson, and Y. Gottlieb. Building a Robust Software-Based Router Using Network Processors. In Proc. of 18th SOSP'01, Chateau Lake Louise, Banff, Canada, Oct. 2001.
[15]
W. Thies, M. Karczmarek, and S. Amarasinghe. StreamIt: A Language for Streaming Applications. In International Conference on Compiler Construction, Apr. 2002.
[16]
D. Xu and X. Jiang. Towards an Integrated Multimedia Service Hosting Overlay. In ACM Multimedia, 2004.
[17]
K. Yocum and J. Chase. Payload Caching: High-Speed Data Forwarding for Network Intermediaries. In Proc. of USENIX Technical Conference, Boston, Massachusetts, June 2001.
[18]
X. Zhang, L. N. Bhuyan, and W.-C. Feng. Anatomy of UDP and M-VIA for Cluster Communications. Journal on Parallel and Distributed Computing, 2005.
[19]
Y. Zhao and R. Storm. Exploiting Event Stream Interpretation in Publish-Subscribe Systems. In ACM Symp. on Principles of Distributed Computing, Aug. 2001.

Cited By

View all
  • (2011)Foresighted tree configuration games in resource constrained distributed stream mining sensorsAd Hoc Networks10.1016/j.adhoc.2010.08.0079:4(497-513)Online publication date: 1-Jun-2011
  • (2007)Advanced networking services for distributed multimedia streaming applicationsMultimedia Tools and Applications10.1007/s11042-006-0089-034:2(179-200)Online publication date: 1-Aug-2007
  • (2005)Addressing data compatibility on programmable network platformsProceedings of the 2005 ACM symposium on Architecture for networking and communications systems10.1145/1095890.1095908(125-134)Online publication date: 26-Oct-2005
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
NOSSDAV '05: Proceedings of the international workshop on Network and operating systems support for digital audio and video
June 2005
210 pages
ISBN:158113987X
DOI:10.1145/1065983
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2005

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. network processors
  2. streaming applications

Qualifiers

  • Article

Conference

NOSSDAV05
Sponsor:

Acceptance Rates

Overall Acceptance Rate 118 of 363 submissions, 33%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 22 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2011)Foresighted tree configuration games in resource constrained distributed stream mining sensorsAd Hoc Networks10.1016/j.adhoc.2010.08.0079:4(497-513)Online publication date: 1-Jun-2011
  • (2007)Advanced networking services for distributed multimedia streaming applicationsMultimedia Tools and Applications10.1007/s11042-006-0089-034:2(179-200)Online publication date: 1-Aug-2007
  • (2005)Addressing data compatibility on programmable network platformsProceedings of the 2005 ACM symposium on Architecture for networking and communications systems10.1145/1095890.1095908(125-134)Online publication date: 26-Oct-2005
  • (2005)C-COREProceedings of the Fourth IEEE International Symposium on Network Computing and Applications10.1109/NCA.2005.10(171-178)Online publication date: 27-Jul-2005

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media