Jeremy Weinberger - Academia.edu (original) (raw)
Uploads
Papers by Jeremy Weinberger
We argue that data webs employing specialized path services, network protocols, and data protocol... more We argue that data webs employing specialized path services, network protocols, and data protocols can be an e#ective platform to analyze and access millions of distributed Gigabyte (and larger) size data sets. We have built a prototype of such a data web today and demonstrated that it can e#ectively access, analyze and mine distributed Gigabyte size data sets even over thousands of miles by using specialized network and data protocols. The prototype uses a server which employs the DataSpace Transfer Protocol or DSTP. Our assumption is that WSDL/SOAP/UDDI-based discovery and description services will enable this same infrastructure to scale to millions of such DSTPServers.
Traditionally, the design and implementation of network based applications, especially large-scal... more Traditionally, the design and implementation of network based applications, especially large-scale, high performance applications, have had to be compromised across multiple dimensions interfaces, services, performance, flexibility, protocols, architecture, technology, etc. These restrictions exist, in part, because the most widely deployed communications infrastructure was designed to optimize traditional communications, not high performance data communications. At iGRID2002, the International Center for Advanced Internet Research (iCAIR) and its research partners demonstrated "Photonic-Empowered Applications," based on next generation intelligent optical networking technology and dynamic data services provisioning. These demonstrations indicated the potential for creating next generation global applications when traditional barriers to network optimization at multiple levels are removed. These application demonstrations were based on high performance communications infra...
20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings., 2003
Summary The OptIPuter is a radical distributed visualization, teleimmersion, data mining and comp... more Summary The OptIPuter is a radical distributed visualization, teleimmersion, data mining and computing architecture. Observing that the exponential growth rates in bandwidth and storage are now much higher than Moore's Law, this major new project of several universities - currently six in the US and one in Amsterdam - exploits a new world of computing in which the central architectural element is optical networking. This transition is caused by the use of parallelism, as in supercomputing a decade ago. However, this time the parallelism is in multiple wavelengths of light, or lambdas, on single optical fibers, creating a LambdaGrid. Providing applications-centric middleware to control the LambdaGrid on a regional and global scale is a key goal of the OptIPuter and StarLight Optical Switching projects. Optical switching, Grid, Optical Grid, LambdaGrid, Control planes, Optical networking, Lightpath provisioning, Dynamic wavelength provisioning, WDM, DWDM, Middleware, OptIPuter, S...
We describe an architecture for next generation, distributed data min- ing systems which integrat... more We describe an architecture for next generation, distributed data min- ing systems which integrates data services to facilitate remote data anal- ysis and distributed data mining, network protocol services for high per- formance data transport, and path services for optical paths. We also present experimental evidence using geoscience data that this architec- ture scales the remote analysis of Gigabyte size data sets over long haul, high performance networks.
IEEE Communications Magazine, 2000
Future Generation Computer Systems, 2003
Future Generation Computer Systems, 2003
Proceedings of the NGDM, Nov 1, 2002
We describe an architecture for next generation, distributed data mining systems which integrates... more We describe an architecture for next generation, distributed data mining systems which integrates data services to facilitate remote data analysis and distributed data mining, network protocol services for high performance data transport, and path services for optical paths. We also present experimental evidence using geoscience data that this architecture scales the remote analysis of Gigabyte size data sets over long haul, high performance networks.
Modern servers pay a heavy price in block access time on disk-bound workloads when the working se... more Modern servers pay a heavy price in block access time on disk-bound workloads when the working set is greater than the size of the local buffer cache. We provide a mechanism for cooperating servers to coordinate and share their local buffer caches. The co-ordinated buffer cache ...
We argue that data webs employing specialized path services, network protocols, and data protocol... more We argue that data webs employing specialized path services, network protocols, and data protocols can be an e#ective platform to analyze and access millions of distributed Gigabyte (and larger) size data sets. We have built a prototype of such a data web today and demonstrated that it can e#ectively access, analyze and mine distributed Gigabyte size data sets even over thousands of miles by using specialized network and data protocols. The prototype uses a server which employs the DataSpace Transfer Protocol or DSTP. Our assumption is that WSDL/SOAP/UDDI-based discovery and description services will enable this same infrastructure to scale to millions of such DSTPServers.
Traditionally, the design and implementation of network based applications, especially large-scal... more Traditionally, the design and implementation of network based applications, especially large-scale, high performance applications, have had to be compromised across multiple dimensions interfaces, services, performance, flexibility, protocols, architecture, technology, etc. These restrictions exist, in part, because the most widely deployed communications infrastructure was designed to optimize traditional communications, not high performance data communications. At iGRID2002, the International Center for Advanced Internet Research (iCAIR) and its research partners demonstrated "Photonic-Empowered Applications," based on next generation intelligent optical networking technology and dynamic data services provisioning. These demonstrations indicated the potential for creating next generation global applications when traditional barriers to network optimization at multiple levels are removed. These application demonstrations were based on high performance communications infra...
20th IEEE/11th NASA Goddard Conference on Mass Storage Systems and Technologies, 2003. (MSST 2003). Proceedings., 2003
Summary The OptIPuter is a radical distributed visualization, teleimmersion, data mining and comp... more Summary The OptIPuter is a radical distributed visualization, teleimmersion, data mining and computing architecture. Observing that the exponential growth rates in bandwidth and storage are now much higher than Moore's Law, this major new project of several universities - currently six in the US and one in Amsterdam - exploits a new world of computing in which the central architectural element is optical networking. This transition is caused by the use of parallelism, as in supercomputing a decade ago. However, this time the parallelism is in multiple wavelengths of light, or lambdas, on single optical fibers, creating a LambdaGrid. Providing applications-centric middleware to control the LambdaGrid on a regional and global scale is a key goal of the OptIPuter and StarLight Optical Switching projects. Optical switching, Grid, Optical Grid, LambdaGrid, Control planes, Optical networking, Lightpath provisioning, Dynamic wavelength provisioning, WDM, DWDM, Middleware, OptIPuter, S...
We describe an architecture for next generation, distributed data min- ing systems which integrat... more We describe an architecture for next generation, distributed data min- ing systems which integrates data services to facilitate remote data anal- ysis and distributed data mining, network protocol services for high per- formance data transport, and path services for optical paths. We also present experimental evidence using geoscience data that this architec- ture scales the remote analysis of Gigabyte size data sets over long haul, high performance networks.
IEEE Communications Magazine, 2000
Future Generation Computer Systems, 2003
Future Generation Computer Systems, 2003
Proceedings of the NGDM, Nov 1, 2002
We describe an architecture for next generation, distributed data mining systems which integrates... more We describe an architecture for next generation, distributed data mining systems which integrates data services to facilitate remote data analysis and distributed data mining, network protocol services for high performance data transport, and path services for optical paths. We also present experimental evidence using geoscience data that this architecture scales the remote analysis of Gigabyte size data sets over long haul, high performance networks.
Modern servers pay a heavy price in block access time on disk-bound workloads when the working se... more Modern servers pay a heavy price in block access time on disk-bound workloads when the working set is greater than the size of the local buffer cache. We provide a mechanism for cooperating servers to coordinate and share their local buffer caches. The co-ordinated buffer cache ...