William Johnston - Academia.edu (original) (raw)

Uploads

Papers by William Johnston

Research paper thumbnail of The advanced networks and services underpinning modern, large-scale science: DOE's ESnet

Grid Computing The New Frontier of High Performance Computing, 2005

Publisher Summary This chapter describes the motivation, architecture, and services of a new netw... more Publisher Summary This chapter describes the motivation, architecture, and services of a new network system. Modern large-scale science requires networking that is global in extent, highly reliable and versatile, and that provides high bandwidth and supports sustained high volume traffic. These requirements resulted in a new approach and architecture for US Department of Energy's Office of Science (DOE's) Energy Sciences Network (ESnet), which is the network that serves all of the major DOE facilities. This new architecture includes elements supporting multiple, high-speed national backbones with different characteristics, redundancy, the quality of service and circuit oriented services, and the interoperation of all of these with the other major national and international networks supporting science. ESnet is driven by the requirements of the science Program Offices in DOE's Office of Science. Several workshops have examined these requirements as they relate to networking and middleware. In the first Office of Science workshop (August, 2002), the goal was to examine the network needs of major Office of Science (OSC) science programs.

Research paper thumbnail of The State of NASA's Information Power Grid

Research paper thumbnail of UNIX based distributed printing in a diverse environment

Experiences using the Berkeley UNIX line printer spooler mechanism to provide distributed, mostly... more Experiences using the Berkeley UNIX line printer spooler mechanism to provide distributed, mostly laser printer based, typesetting and graphics output to a geographically dispersed, heterogeneous, set of host computers and users are described. The user interface is the usual set of UNIX commands, though the methodology employed is somewhat different from the usual. The user's environment is resolved on the local machine and the tasks of document formatting and device driving are relegated to dedicated server systems in order to remove these computer intensive tasks from the timesharing client, or user, systems. The details of the system are described, together with an analysis of performance issues and operational aspects.

Research paper thumbnail of Chapter 27 Implementing Production Grids for Science and Engineering

We examine experience accrued by those who have been involved with building large-scale Grids int... more We examine experience accrued by those who have been involved with building large-scale Grids intended for production use. By this we mean that the primary task of such Grids is to enable largescale scientific research rather than to act as a test-bed for research into software development, or more speculative computer science. Severe constraints are brought to bear in the construction of such Grids, since access to such resources is subject to stringent requirements of security and high quality of service. We must also reckon with the fact that the resources on such Grids may not be primarily designed or tailored to Grid use. The Grid middleware cannot control policy on such Grids, it must co-operate with the site policy and resource management systems. We may be dealing with ancillary resources, such as telescopes and experimental facilities, that are not computational, may not possess a full operating system, and may have policy and management requirements that are very different...

Research paper thumbnail of Information Rich GIS Dissemination in Disconnected Environments

Transactions in GIS, 2013

Research paper thumbnail of NASA's Information Power Grid: Distributed High-Performance Computing and Large-Scale Data Management for Science and Engineering

… l Conference onComputing …, 2000

... These includ the toolkits for construction of application frameworks / problem solving enviro... more ... These includ the toolkits for construction of application frameworks / problem solving environments (PSE ... Operational documentation and procedures are essential to managing the Grid as a ... control model that provides fo management of stakeholder rights (use-conditions) and ...

Research paper thumbnail of Distributed Large Data-Object Environments

ABSTRACT We have developed and deployed a Distributed-Parallel Storage System (DPSS) in several h... more ABSTRACT We have developed and deployed a Distributed-Parallel Storage System (DPSS) in several high speed ATM WAN testbeds to support several different types of data-intensive applications. Architecturally the DPSS is a network striped disk array, but is fairly unique in that its implementation allows applications complete freedom to determine optimal data layout, replication and/or coding redundancy strategy, security policy, and dynamic reconfiguration. In conjunction with the DPSS, we have developed a "top-to-bottom, end-to-end" performance monitoring and analysis methodology that has allowed us to characterize all aspects of the DPSS operating in high speed WAN environments. In particular, we have run a variety of performance monitoring experiments involving the DPSS in the MAGIC testbed, which is a large-scale, high-speed, ATM network and we describe our experience using the monitoring methodology to identify and correcting problems that limit the performance of high speed distributed applications.

Research paper thumbnail of Rationale and Strategy for a 21st Century Scientific Computing Architecture: The Case for using Commercial Symmetric Multiprocessors as Supercomputers

International Journal of High Speed Computing, 1997

Research paper thumbnail of ∗Energy Sciences Network

— With the advent of service sensitive applications such as remote controlled experiments, time c... more — With the advent of service sensitive applications such as remote controlled experiments, time constrained massive data transfers, and video-conferencing, it has become apparent that there is a need for the setup of dynamically provisioned, quality of service enabled virtual circuits. The ESnet On-Demand Secure Circuits and Advance Reservation System (OSCARS) is a prototype service enabling advance reservation of guaranteed bandwidth secure virtual circuits. OSCARS operates within the Energy Sciences Network (ESnet), and has provisions for interoperation with other network domains. ESnet is a high-speed network serving thousands of Department of Energy scientists and collaborators worldwide. OSCARS utilizes the Web services model and standards to implement communication with the system and between domains, and for authentication, authorization, and auditing (AAA). The management and operation of end-to-end virtual circuits within the network is done at the layer 3 network level. Mu...

Research paper thumbnail of QoS as middleware: bandwidth reservation system design

We wish to provide quality of service (QoS) to scientists who are remotely controlling experiment... more We wish to provide quality of service (QoS) to scientists who are remotely controlling experiments on singular instruments such as the LBL Advanced Light Source (ALS) or who need to harness heterogeneous and distributed computing resources to perform large-scale computation. To ensure the timely availability of the computing, storage, and networking resources required for data collection and/or analysis, use of these resources must be scheduled in advance. The bandwidth reservation system (BRS) reserves time in IP differentiated service classes (S. Blake et al.). These classes have upper bounds on the total bandwidth allocation. The reservation unit is a slot: a well-defined period of time with an associated bandwidth. The sum of the allocated bandwidths in all of the slot allocations never exceeds the maximum bandwidth defined for the service class. The result is that the classes are never oversubscribed

Research paper thumbnail of (From the FY2008 Budget Presentation of

Research paper thumbnail of Efficient and dependable multimedia data delivery service in World Wide Web environment

Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. HICSS-32. Abstracts and CD-ROM of Full Papers, 1999

Research paper thumbnail of Using Computing and Data Grids for Large-Scale Science and Engineering

International Journal of High Performance Computing Applications, 2001

Research paper thumbnail of Enabling high throughput in widely distributed data management and analysis systems: Lessons from the LHC

Research paper thumbnail of Prague, Czech Republic Motivation, Design, Deployment and Evolution of a Guaranteed Bandwidth Network Service

Research paper thumbnail of Energy Sciences Network

Research paper thumbnail of Enabling high throughput in widely distributed data management and analysis systems: Lessons from the LHC

Research paper thumbnail of The Evolution Towards Grids: Ten Years of High-Speed, Wide Area, Data Intensive Computing

Modern scientific computing involves organizing, moving, visualizing, and analyzing massive amoun... more Modern scientific computing involves organizing, moving, visualizing, and analyzing massive amounts of data from around the world, as well as employing large-scale computation. The distributed systems that solve large-scale problems will always involve aggregating and scheduling many resources. Data must be located and staged, cache and network capacity must be available at the same time as computing capacity, etc. Every aspect of such a system is dynamic: locating and scheduling resources, adapting running application systems to availability and congestion in the middleware and infrastructure, responding to human interaction, etc. The technologies, the middleware services, and the architectures that are used to build useful high-speed, wide area distributed systems, are now being integrated in “Grids ” [1]. This paper explores some of the background, current state, and future directions of Grids. 1

Research paper thumbnail of Networking for the Future of Science

Research paper thumbnail of Using the World Wide Web to provide a platform independent interface to high performance computing

We have developed a set of techniques for providing interactive 3D graphics via the World Wide We... more We have developed a set of techniques for providing interactive 3D graphics via the World Wide Web (WWW) as part of the " Whole Frog" project. The success of this project indicates that the Web and its associated browsers can serve as an easily used and powerful front end to high-performance computing resources. We utilize the Common Gateway Interface capability of WWW servers to provide an interactive 3D visualization front end through Web clients. These techniques have been used to make a "Virtual Frog Dissection Kit", available as http://www-itg.lbl.gov/vfrog. A student using this Kit has the ability to view various parts of a frog from many different angles, and with the different anatomical structures visible or invisible.

Research paper thumbnail of The advanced networks and services underpinning modern, large-scale science: DOE's ESnet

Grid Computing The New Frontier of High Performance Computing, 2005

Publisher Summary This chapter describes the motivation, architecture, and services of a new netw... more Publisher Summary This chapter describes the motivation, architecture, and services of a new network system. Modern large-scale science requires networking that is global in extent, highly reliable and versatile, and that provides high bandwidth and supports sustained high volume traffic. These requirements resulted in a new approach and architecture for US Department of Energy's Office of Science (DOE's) Energy Sciences Network (ESnet), which is the network that serves all of the major DOE facilities. This new architecture includes elements supporting multiple, high-speed national backbones with different characteristics, redundancy, the quality of service and circuit oriented services, and the interoperation of all of these with the other major national and international networks supporting science. ESnet is driven by the requirements of the science Program Offices in DOE's Office of Science. Several workshops have examined these requirements as they relate to networking and middleware. In the first Office of Science workshop (August, 2002), the goal was to examine the network needs of major Office of Science (OSC) science programs.

Research paper thumbnail of The State of NASA's Information Power Grid

Research paper thumbnail of UNIX based distributed printing in a diverse environment

Experiences using the Berkeley UNIX line printer spooler mechanism to provide distributed, mostly... more Experiences using the Berkeley UNIX line printer spooler mechanism to provide distributed, mostly laser printer based, typesetting and graphics output to a geographically dispersed, heterogeneous, set of host computers and users are described. The user interface is the usual set of UNIX commands, though the methodology employed is somewhat different from the usual. The user's environment is resolved on the local machine and the tasks of document formatting and device driving are relegated to dedicated server systems in order to remove these computer intensive tasks from the timesharing client, or user, systems. The details of the system are described, together with an analysis of performance issues and operational aspects.

Research paper thumbnail of Chapter 27 Implementing Production Grids for Science and Engineering

We examine experience accrued by those who have been involved with building large-scale Grids int... more We examine experience accrued by those who have been involved with building large-scale Grids intended for production use. By this we mean that the primary task of such Grids is to enable largescale scientific research rather than to act as a test-bed for research into software development, or more speculative computer science. Severe constraints are brought to bear in the construction of such Grids, since access to such resources is subject to stringent requirements of security and high quality of service. We must also reckon with the fact that the resources on such Grids may not be primarily designed or tailored to Grid use. The Grid middleware cannot control policy on such Grids, it must co-operate with the site policy and resource management systems. We may be dealing with ancillary resources, such as telescopes and experimental facilities, that are not computational, may not possess a full operating system, and may have policy and management requirements that are very different...

Research paper thumbnail of Information Rich GIS Dissemination in Disconnected Environments

Transactions in GIS, 2013

Research paper thumbnail of NASA's Information Power Grid: Distributed High-Performance Computing and Large-Scale Data Management for Science and Engineering

… l Conference onComputing …, 2000

... These includ the toolkits for construction of application frameworks / problem solving enviro... more ... These includ the toolkits for construction of application frameworks / problem solving environments (PSE ... Operational documentation and procedures are essential to managing the Grid as a ... control model that provides fo management of stakeholder rights (use-conditions) and ...

Research paper thumbnail of Distributed Large Data-Object Environments

ABSTRACT We have developed and deployed a Distributed-Parallel Storage System (DPSS) in several h... more ABSTRACT We have developed and deployed a Distributed-Parallel Storage System (DPSS) in several high speed ATM WAN testbeds to support several different types of data-intensive applications. Architecturally the DPSS is a network striped disk array, but is fairly unique in that its implementation allows applications complete freedom to determine optimal data layout, replication and/or coding redundancy strategy, security policy, and dynamic reconfiguration. In conjunction with the DPSS, we have developed a "top-to-bottom, end-to-end" performance monitoring and analysis methodology that has allowed us to characterize all aspects of the DPSS operating in high speed WAN environments. In particular, we have run a variety of performance monitoring experiments involving the DPSS in the MAGIC testbed, which is a large-scale, high-speed, ATM network and we describe our experience using the monitoring methodology to identify and correcting problems that limit the performance of high speed distributed applications.

Research paper thumbnail of Rationale and Strategy for a 21st Century Scientific Computing Architecture: The Case for using Commercial Symmetric Multiprocessors as Supercomputers

International Journal of High Speed Computing, 1997

Research paper thumbnail of ∗Energy Sciences Network

— With the advent of service sensitive applications such as remote controlled experiments, time c... more — With the advent of service sensitive applications such as remote controlled experiments, time constrained massive data transfers, and video-conferencing, it has become apparent that there is a need for the setup of dynamically provisioned, quality of service enabled virtual circuits. The ESnet On-Demand Secure Circuits and Advance Reservation System (OSCARS) is a prototype service enabling advance reservation of guaranteed bandwidth secure virtual circuits. OSCARS operates within the Energy Sciences Network (ESnet), and has provisions for interoperation with other network domains. ESnet is a high-speed network serving thousands of Department of Energy scientists and collaborators worldwide. OSCARS utilizes the Web services model and standards to implement communication with the system and between domains, and for authentication, authorization, and auditing (AAA). The management and operation of end-to-end virtual circuits within the network is done at the layer 3 network level. Mu...

Research paper thumbnail of QoS as middleware: bandwidth reservation system design

We wish to provide quality of service (QoS) to scientists who are remotely controlling experiment... more We wish to provide quality of service (QoS) to scientists who are remotely controlling experiments on singular instruments such as the LBL Advanced Light Source (ALS) or who need to harness heterogeneous and distributed computing resources to perform large-scale computation. To ensure the timely availability of the computing, storage, and networking resources required for data collection and/or analysis, use of these resources must be scheduled in advance. The bandwidth reservation system (BRS) reserves time in IP differentiated service classes (S. Blake et al.). These classes have upper bounds on the total bandwidth allocation. The reservation unit is a slot: a well-defined period of time with an associated bandwidth. The sum of the allocated bandwidths in all of the slot allocations never exceeds the maximum bandwidth defined for the service class. The result is that the classes are never oversubscribed

Research paper thumbnail of (From the FY2008 Budget Presentation of

Research paper thumbnail of Efficient and dependable multimedia data delivery service in World Wide Web environment

Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. HICSS-32. Abstracts and CD-ROM of Full Papers, 1999

Research paper thumbnail of Using Computing and Data Grids for Large-Scale Science and Engineering

International Journal of High Performance Computing Applications, 2001

Research paper thumbnail of Enabling high throughput in widely distributed data management and analysis systems: Lessons from the LHC

Research paper thumbnail of Prague, Czech Republic Motivation, Design, Deployment and Evolution of a Guaranteed Bandwidth Network Service

Research paper thumbnail of Energy Sciences Network

Research paper thumbnail of Enabling high throughput in widely distributed data management and analysis systems: Lessons from the LHC

Research paper thumbnail of The Evolution Towards Grids: Ten Years of High-Speed, Wide Area, Data Intensive Computing

Modern scientific computing involves organizing, moving, visualizing, and analyzing massive amoun... more Modern scientific computing involves organizing, moving, visualizing, and analyzing massive amounts of data from around the world, as well as employing large-scale computation. The distributed systems that solve large-scale problems will always involve aggregating and scheduling many resources. Data must be located and staged, cache and network capacity must be available at the same time as computing capacity, etc. Every aspect of such a system is dynamic: locating and scheduling resources, adapting running application systems to availability and congestion in the middleware and infrastructure, responding to human interaction, etc. The technologies, the middleware services, and the architectures that are used to build useful high-speed, wide area distributed systems, are now being integrated in “Grids ” [1]. This paper explores some of the background, current state, and future directions of Grids. 1

Research paper thumbnail of Networking for the Future of Science

Research paper thumbnail of Using the World Wide Web to provide a platform independent interface to high performance computing

We have developed a set of techniques for providing interactive 3D graphics via the World Wide We... more We have developed a set of techniques for providing interactive 3D graphics via the World Wide Web (WWW) as part of the " Whole Frog" project. The success of this project indicates that the Web and its associated browsers can serve as an easily used and powerful front end to high-performance computing resources. We utilize the Common Gateway Interface capability of WWW servers to provide an interactive 3D visualization front end through Web clients. These techniques have been used to make a "Virtual Frog Dissection Kit", available as http://www-itg.lbl.gov/vfrog. A student using this Kit has the ability to view various parts of a frog from many different angles, and with the different anatomical structures visible or invisible.