Raquel Vigolvino Lopes - Academia.edu (original) (raw)
Uploads
Papers by Raquel Vigolvino Lopes
The new ways of doing science rooted on the unprecedented processing, communication, and storage ... more The new ways of doing science rooted on the unprecedented processing, communication, and storage infrastructure that became available to scientists are collectively called e-Science. Due to their nature, most e-Science activities can only be successfully performed if researchers have access to high-performance computing facilities. Grid and voluntary computing are well-established solutions that cater to this need, but are not accessible to all labs and institutions. Peer-to-peer (P2P) grid computing has been proposed to address this very problem. In this chapter, we share our experience in developing a P2P grid middleware called OurGrid and deploying it to build the OurGrid Community. We describe the mechanisms that effectively promote collaboration and allow the assemblage of large P2P grids from the contributions of thousands of small sites. This includes a thorough review of the main mechanisms required to support the execution of bag-of-tasks applications on top of P2P grids: accounting, scheduling, security, and data caching. Besides, we discuss ways to allow P2P grids to interoperate with service grids. We also report a success case in the utilization of the OurGrid middleware in the context of e-Science. Finally, we summarize our experience in this area indicating the lessons we have learned, the present challenges, and future directions of research.
Bookmarks Related papers MentionsView impact
The cloud computing market has emerged as an alternative for the provisioning of resources on a p... more The cloud computing market has emerged as an alternative for the provisioning of resources on a pay-as-you-go basis. This flexibility potentially allows clients of cloud computing solutions to reduce the total cost of ownership of their Information Technology infrastructures. On the other hand, this market-based model is not the only way to reduce costs. Among other solutions proposed, peer-to-peer (P2P) grid computing has been suggested as a way to enable a simpler economy for the trading of idle resources. In this paper, we consider an IT infrastructure which benefits from both of these strategies. In such a hybrid infrastructure, computing power can be obtained from in-house dedicated resources, from resources acquired from cloud computing providers, and from resources received as donations from a P2P grid. We take a business-driven approach to the problem and try to maximise the profit that can be achieved by running applications in this hybrid infrastructure. The execution of applications yields utility, while costs may be incurred when resources are used to run the applications, or even when they sit idle. We assume that resources made available from cloud computing providers can be either reserved in advance, or bought on-demand. We study the impact that long-term contracts established with the cloud computing providers have on the profit achieved. Anticipating the optimal contracts is not possible due to the many uncertainties in the system, which stem from the prediction error on the workload demand, the lack of guarantees on the quality of service of the P2P grid, and fluctuations in the future prices of on-demand resources. However, we show that the judicious planning of long term contracts can lead to profits close to those given by an optimal contract set. In particular, we model the planning problem as an optimisation problem and show that the planning performed by solving this optimization problem is robust to the inherent uncertainties of the system,- - producing profits that for some scenarios can be more than double those achieved by following some common rule-of-thumb approaches to choosing reservation contracts.
Bookmarks Related papers MentionsView impact
The energy costs of running computer systems are a growing concern: for large data centers, recen... more The energy costs of running computer systems are a growing concern: for large data centers, recent estimates put these costs higher than the cost of hardware itself. As a consequence, energy efficiency has become a pervasive theme for designing, deploying, and operating computer systems. This paper evaluates the energy trade-offs brought by data deduplication in distributed storage systems. Depending on the workload, deduplication can enable a lower storage footprint, reduce the I/O pressure on the storage system, and reduce network traffic, at the cost of increased computational overhead. From an energy perspective, data deduplication enables a trade-off between the energy consumed for additional computation and the energy saved by lower storage and network load. The main point our experiments and model bring home is the following: while for non energy-proportional machines performance- and energy-centric optimizations have break-even points that are relatively close, for the newer generation of energy proportional machines the break-even points are significantly different. An important consequence of this difference is that, with newer systems, there are higher energy inefficiencies when the system is optimized for performance.
Bookmarks Related papers MentionsView impact
Load variations are unexpected perturbations that can degrade performance or even cause unavailab... more Load variations are unexpected perturbations that can degrade performance or even cause unavailability of a system. There are efforts that attempt to dynamically provide resources to accommodate load fluctuations during the execution of applications. However, these efforts do not consider the existence of software faults, whose effects can influence the application behavior and its quality of service, and may mislead a dynamic provisioning system. When trying to tackle both problems simultaneously the fundamental issue to be addressed is how to differentiate a saturated application from a faulty one. The contributions of this paper are threefold. Firstly, we introduce the idea of taking software faults into account when specifying a dynamic provisioning scheme. Secondly, we define a simple algorithm that can be used to distinguish saturated from faulty software. By implementing this algorithm one is able to realize dynamic provisioning with restarts into a full server infrastructure data center. Finally, we implement this algorithm and experimentally demonstrate its efficacy.
Bookmarks Related papers MentionsView impact
The new ways of doing science, rooted on the unprecedented processing, communication and storage ... more The new ways of doing science, rooted on the unprecedented processing, communication and storage infrastructures that became available to scientists, are collectively called e-Science. Many research labs now need non-trivial computational power to run e-Science applications. Grid and voluntary computing are well-established solutions that cater to this need, but are not accessible for all labs and institutions. Besides, there is an uncertainty about the future amount of resources that will be available in such infrastructures, which prevents the researchers from planning their activities to guarantee that deadlines will be met. With the emergence of the cloud computing paradigm come new opportunities. One possibility is to run e-Science activities at resources acquired on-demand from cloud providers. However, although very low, there is a cost associated with the usage of cloud resources. Besides that, the amount of resources that can be simultaneously acquired is, in practice, limited. Another possibility is the not new idea of composing hybrid infrastructures in which the huge amount of computational resources shared by the grid infrastructures are used whenever possible and extra capacity is acquired from cloud computing providers. We here investigate how to schedule e-Science activities in such hybrid infrastructures so that deadlines are met and costs are reduced.
Bookmarks Related papers MentionsView impact
Dynamic provisioning systems change application capacity in order to use enough resources to acco... more Dynamic provisioning systems change application capacity in order to use enough resources to accommodate current load. Rejuvenation systems detect/forecast software failures and temporarily remove one or more components of the application in order to bring them to a clean state. Up to now, these systems have been developed unaware of one another. However, many applications need to be controlled by both. In this paper we investigate whether these systems can actuate over the same application when they are not aware of each other, i.e., without coordination. We present and apply a model to study the performance of dynamic provisioning and rejuvenation systems when they actuate over the same application without coordination. Our results show that when both systems coexist application quality of service degrades in comparison with the quality of service provided when each system is acting alone. This suggests that some level of coordination must be added to maximize the benefits gained from the simultaneous use of both systems.
Bookmarks Related papers MentionsView impact
With the emergence of the cloud computing paradigm and the continuous search to reduce the cost o... more With the emergence of the cloud computing paradigm and the continuous search to reduce the cost of running Information Technology (IT) infrastructures, we are currently experiencing an important change in the way these infrastructures are assembled, configured and managed. In this research we consider the problem of managing a computing infrastructure whose processing elements are acquired from infrastructure-as-a-service (IaaS) providers, and used to support the execution of long-lived on-line transactions processing applications, whose workloads experience huge fluctuations over time, such as Web applications. Resources can be acquired from IaaS providers in different ways, each with its own pricing scheme and associated quality of service. One acquisition model allows resources to be reserved for long periods of usage at a reduced usage price, while others allow dedicated resources to be instantiated on demand at any time, subject to the availability of resources, with a usage price that is usually larger than that paid for reserved resources. In this context, the problem that we address in this paper is how the provider of a Web application should plan the long-term reservation contracts with an IaaS provider, in such a way that its profitability is increased. We propose a model that can be used for guiding this capacity planning activity. We use the model to evaluate the gains that can be achieved with the judicious planning of the infrastructure capacity in a number of scenarios. We show that the gains can be substantial, specially when the load variation between normal operational periods and surge periods is large.
Bookmarks Related papers MentionsView impact
Peer-to-peer (P2P) desktop grids have been proposed as an economical way to increase the processi... more Peer-to-peer (P2P) desktop grids have been proposed as an economical way to increase the processing capabilities of information technology (IT) infrastructures. In a P2P grid, a peer donates its idle resources to the other peers in the system, and, in exchange, can use the idle resources of other peers when its processing demand surpasses its local computing capacity. Despite their cost-effectiveness, scheduling of processing demands on IT infrastructures that encompass P2P desktop grids is more difficult. At the root of this difficulty is the fact that the quality of the service provided by P2P desktop grids varies significantly over time. The research we report in this paper tackles the problem of estimating the quality of service of P2P desktop grids. We base our study on the OurGrid system, which implements an autonomous incentive mechanism based on reciprocity, called the Network of Favours (NoF). In this paper we propose a model for predicting the quality of service of a P2P desktop grid that uses the NoF incentive mechanism. The model proposed is able to estimate the amount of resources that is available for a peer in the system at future instants of time. We also evaluate the accuracy of the model by running simulation experiments fed with field data. Our results show that in the worst scenario the proposed model is able to predict how much of a given demand for resources a peer is going to obtain from the grid with a mean prediction error of only 7.2%.
Bookmarks Related papers MentionsView impact
The new ways of doing science rooted on the unprecedented processing, communication, and storage ... more The new ways of doing science rooted on the unprecedented processing, communication, and storage infrastructure that became available to scientists are collectively called e-Science. Due to their nature, most e-Science activities can only be successfully performed if researchers have access to high-performance computing facilities. Grid and voluntary computing are well-established solutions that cater to this need, but are not accessible to all labs and institutions. Peer-to-peer (P2P) grid computing has been proposed to address this very problem. In this chapter, we share our experience in developing a P2P grid middleware called OurGrid and deploying it to build the OurGrid Community. We describe the mechanisms that effectively promote collaboration and allow the assemblage of large P2P grids from the contributions of thousands of small sites. This includes a thorough review of the main mechanisms required to support the execution of bag-of-tasks applications on top of P2P grids: accounting, scheduling, security, and data caching. Besides, we discuss ways to allow P2P grids to interoperate with service grids. We also report a success case in the utilization of the OurGrid middleware in the context of e-Science. Finally, we summarize our experience in this area indicating the lessons we have learned, the present challenges, and future directions of research.
Bookmarks Related papers MentionsView impact
The cloud computing market has emerged as an alternative for the provisioning of resources on a p... more The cloud computing market has emerged as an alternative for the provisioning of resources on a pay-as-you-go basis. This flexibility potentially allows clients of cloud computing solutions to reduce the total cost of ownership of their Information Technology infrastructures. On the other hand, this market-based model is not the only way to reduce costs. Among other solutions proposed, peer-to-peer (P2P) grid computing has been suggested as a way to enable a simpler economy for the trading of idle resources. In this paper, we consider an IT infrastructure which benefits from both of these strategies. In such a hybrid infrastructure, computing power can be obtained from in-house dedicated resources, from resources acquired from cloud computing providers, and from resources received as donations from a P2P grid. We take a business-driven approach to the problem and try to maximise the profit that can be achieved by running applications in this hybrid infrastructure. The execution of applications yields utility, while costs may be incurred when resources are used to run the applications, or even when they sit idle. We assume that resources made available from cloud computing providers can be either reserved in advance, or bought on-demand. We study the impact that long-term contracts established with the cloud computing providers have on the profit achieved. Anticipating the optimal contracts is not possible due to the many uncertainties in the system, which stem from the prediction error on the workload demand, the lack of guarantees on the quality of service of the P2P grid, and fluctuations in the future prices of on-demand resources. However, we show that the judicious planning of long term contracts can lead to profits close to those given by an optimal contract set. In particular, we model the planning problem as an optimisation problem and show that the planning performed by solving this optimization problem is robust to the inherent uncertainties of the system,- - producing profits that for some scenarios can be more than double those achieved by following some common rule-of-thumb approaches to choosing reservation contracts.
Bookmarks Related papers MentionsView impact
The energy costs of running computer systems are a growing concern: for large data centers, recen... more The energy costs of running computer systems are a growing concern: for large data centers, recent estimates put these costs higher than the cost of hardware itself. As a consequence, energy efficiency has become a pervasive theme for designing, deploying, and operating computer systems. This paper evaluates the energy trade-offs brought by data deduplication in distributed storage systems. Depending on the workload, deduplication can enable a lower storage footprint, reduce the I/O pressure on the storage system, and reduce network traffic, at the cost of increased computational overhead. From an energy perspective, data deduplication enables a trade-off between the energy consumed for additional computation and the energy saved by lower storage and network load. The main point our experiments and model bring home is the following: while for non energy-proportional machines performance- and energy-centric optimizations have break-even points that are relatively close, for the newer generation of energy proportional machines the break-even points are significantly different. An important consequence of this difference is that, with newer systems, there are higher energy inefficiencies when the system is optimized for performance.
Bookmarks Related papers MentionsView impact
Load variations are unexpected perturbations that can degrade performance or even cause unavailab... more Load variations are unexpected perturbations that can degrade performance or even cause unavailability of a system. There are efforts that attempt to dynamically provide resources to accommodate load fluctuations during the execution of applications. However, these efforts do not consider the existence of software faults, whose effects can influence the application behavior and its quality of service, and may mislead a dynamic provisioning system. When trying to tackle both problems simultaneously the fundamental issue to be addressed is how to differentiate a saturated application from a faulty one. The contributions of this paper are threefold. Firstly, we introduce the idea of taking software faults into account when specifying a dynamic provisioning scheme. Secondly, we define a simple algorithm that can be used to distinguish saturated from faulty software. By implementing this algorithm one is able to realize dynamic provisioning with restarts into a full server infrastructure data center. Finally, we implement this algorithm and experimentally demonstrate its efficacy.
Bookmarks Related papers MentionsView impact
The new ways of doing science, rooted on the unprecedented processing, communication and storage ... more The new ways of doing science, rooted on the unprecedented processing, communication and storage infrastructures that became available to scientists, are collectively called e-Science. Many research labs now need non-trivial computational power to run e-Science applications. Grid and voluntary computing are well-established solutions that cater to this need, but are not accessible for all labs and institutions. Besides, there is an uncertainty about the future amount of resources that will be available in such infrastructures, which prevents the researchers from planning their activities to guarantee that deadlines will be met. With the emergence of the cloud computing paradigm come new opportunities. One possibility is to run e-Science activities at resources acquired on-demand from cloud providers. However, although very low, there is a cost associated with the usage of cloud resources. Besides that, the amount of resources that can be simultaneously acquired is, in practice, limited. Another possibility is the not new idea of composing hybrid infrastructures in which the huge amount of computational resources shared by the grid infrastructures are used whenever possible and extra capacity is acquired from cloud computing providers. We here investigate how to schedule e-Science activities in such hybrid infrastructures so that deadlines are met and costs are reduced.
Bookmarks Related papers MentionsView impact
Dynamic provisioning systems change application capacity in order to use enough resources to acco... more Dynamic provisioning systems change application capacity in order to use enough resources to accommodate current load. Rejuvenation systems detect/forecast software failures and temporarily remove one or more components of the application in order to bring them to a clean state. Up to now, these systems have been developed unaware of one another. However, many applications need to be controlled by both. In this paper we investigate whether these systems can actuate over the same application when they are not aware of each other, i.e., without coordination. We present and apply a model to study the performance of dynamic provisioning and rejuvenation systems when they actuate over the same application without coordination. Our results show that when both systems coexist application quality of service degrades in comparison with the quality of service provided when each system is acting alone. This suggests that some level of coordination must be added to maximize the benefits gained from the simultaneous use of both systems.
Bookmarks Related papers MentionsView impact
With the emergence of the cloud computing paradigm and the continuous search to reduce the cost o... more With the emergence of the cloud computing paradigm and the continuous search to reduce the cost of running Information Technology (IT) infrastructures, we are currently experiencing an important change in the way these infrastructures are assembled, configured and managed. In this research we consider the problem of managing a computing infrastructure whose processing elements are acquired from infrastructure-as-a-service (IaaS) providers, and used to support the execution of long-lived on-line transactions processing applications, whose workloads experience huge fluctuations over time, such as Web applications. Resources can be acquired from IaaS providers in different ways, each with its own pricing scheme and associated quality of service. One acquisition model allows resources to be reserved for long periods of usage at a reduced usage price, while others allow dedicated resources to be instantiated on demand at any time, subject to the availability of resources, with a usage price that is usually larger than that paid for reserved resources. In this context, the problem that we address in this paper is how the provider of a Web application should plan the long-term reservation contracts with an IaaS provider, in such a way that its profitability is increased. We propose a model that can be used for guiding this capacity planning activity. We use the model to evaluate the gains that can be achieved with the judicious planning of the infrastructure capacity in a number of scenarios. We show that the gains can be substantial, specially when the load variation between normal operational periods and surge periods is large.
Bookmarks Related papers MentionsView impact
Peer-to-peer (P2P) desktop grids have been proposed as an economical way to increase the processi... more Peer-to-peer (P2P) desktop grids have been proposed as an economical way to increase the processing capabilities of information technology (IT) infrastructures. In a P2P grid, a peer donates its idle resources to the other peers in the system, and, in exchange, can use the idle resources of other peers when its processing demand surpasses its local computing capacity. Despite their cost-effectiveness, scheduling of processing demands on IT infrastructures that encompass P2P desktop grids is more difficult. At the root of this difficulty is the fact that the quality of the service provided by P2P desktop grids varies significantly over time. The research we report in this paper tackles the problem of estimating the quality of service of P2P desktop grids. We base our study on the OurGrid system, which implements an autonomous incentive mechanism based on reciprocity, called the Network of Favours (NoF). In this paper we propose a model for predicting the quality of service of a P2P desktop grid that uses the NoF incentive mechanism. The model proposed is able to estimate the amount of resources that is available for a peer in the system at future instants of time. We also evaluate the accuracy of the model by running simulation experiments fed with field data. Our results show that in the worst scenario the proposed model is able to predict how much of a given demand for resources a peer is going to obtain from the grid with a mean prediction error of only 7.2%.
Bookmarks Related papers MentionsView impact