Improving resource utilisation in market oriented grid management and scheduling (original) (raw)

Market-based grid resource co-allocation and reservation for applications with hard deadlines

Concurrency and Computation: Practice and Experience, 2009

Grid computing technology enables the creation of large-scale IT infrastructures that are shared across organizational boundaries. In such shared infrastructures, conflicting user requirements are common and originate from the selfish actions users perform when formulating their service requests. The introduction of economic principles in grid resource management offers a promising way of dealing with these conflicts. We develop and analyze both a centralized and a decentralized algorithm for economic grid resource management in the context of CPU bound applications with deadlinebased QoS requirements and non-migratable workloads. Through the use of reservations we co-allocate resources across multiple providers in order to ensure that applications finish within their deadline. An evaluation of both algorithms is presented and their performance in terms of realized user value is compared to a non-economic resource manager and to an existing market-based resource management algorithm. We establish that our algorithms perform well and quantify the effect of allowing local workload preemption and different scheduling heuristics on realized user value.

An Economic Model for Grid Scheduling

2007

Grid scheduling, that is, the allocation of distributed computational resource s to user applications, is one of the most challenging and complex task in Grid computing. In this paper, we give a quantitative description of a tender/contract-net model. The performance of the proposed market-based approach is experimentally compared with a simple round-robin allocation protocol.

Resource Brokering in Grid Computing

2018

Ontario A thesis submitted in partial fulfillment of the requirements for the Master of Engineering Science degree in Electrical and Computer Engineering

Capacity planning and scheduling in Grid computing environments

Future Generation Computer Systems - FGCS, 2008

Grid computing infrastructures embody a cost-effective computing paradigm that virtualises heterogeneous system resources to meet the dynamic needs of critical business and scientific applications. These applications range from batch processes and long-running tasks to real-time and even transactional applications. Grid computing environments are inherently dynamic and unpredictable environments sharing services amongst many different users. Grid schedulers aim to make the most efficient use of Grid resources (high utilisation) while providing the best possible performance to the Grid applications (reducing makespan) and satisfying the associated performance and Quality of Service (QoS) constraints. Additionally, in commercial Grid settings where economic considerations are an increasingly important part of Grid scheduling, it is necessary to minimise the cost of application execution on the behalf of the Grid users while ensuring that the applications meet their QoS constraints. Furthermore, efficient resource allocation may allow a resource broker to maximise their profit by minimising the quantity of resource procurement. Scheduling in such a large-scale, dynamic and distributed environment is a complex undertaking. In this paper, we propose an approach to Grid scheduling which abstracts over the details of individual applications, focusing instead on the global cost optimisation problem while taking into account the entire workload, dynamically adjusting to the varying service demands. Our model places particular emphasis on the stochastic and unpredictable nature of the Grid, leading to a more accurate reflection of the state of the Grid and hence more efficient and accurate scheduling decisions.

A Decentralized Grid Market Infrastructure for Service Oriented Grids

2008

Abstract. Service Oriented Computing has a deep impact on how IT infrastructures are conceived both in academia (e-science) and in industry (Service Oriented Architecture and commercial Web Services). Increasingly, economic models are being considered as suitable coordination mechanism for the management of service allocations to clients. However, few complete infrastructures have demonstrated the enabling of economicbased Service Oriented Grids (SOGs).

Using clouds to scale grid resources: An economic model

Future Generation Computer Systems, 2012

Infrastructure as a Service clouds are a flexible and fast way to obtain (virtual) resources as demand varies. Grids, on the other hand, are middleware platforms able to combine resources from different administrative domains for task execution. Clouds can be used by grids as providers of devices such as virtual machines, so they only use the resources they need. But this requires grids to be able to decide when to allocate and release those resources. Here we introduce and analyze by simulations an economic mechanism (a) to set resource prices and (b) resolve when to scale resources depending on the users' demand. This system has a strong emphasis on fairness, so no user hinders the execution of other users' tasks by getting too many resources.

Dispelling Seven Myths about Grid Resource Management

Grid resource management is often viewed as scheduling for large, long-running, compute-intensive, parallel applications. This view is narrow, as we will argue in this article. Grids today encompass diverse resources including machines, users and applications. Moreover, grid users make different demands from different resources. Therefore, grid infrastructures must adopt a greater responsibility than before for managing resources. Grid resource management cannot mean just scheduling jobs on the fastest machines, but must also include scheduling special jobs on matched machines, preserving site autonomy, determining usage policies, respecting permissions for use and so on. In this article, we will present seven "myths" or common beliefs about grid resource management, and dispel each myth by presenting counterexamples or "observations". These observations have been culled from our experience as well as work done by several experts in major grid-related projects. In order to relate these observations to concrete implementation, we will also present grid resource management in Legion, a grid infrastructure project initiated at the University of Virginia. Grids are collections of interconnected resources harnessed together in order to satisfy various needs of users. The resources may be administered by different organisations and may be widely-distributed, heterogeneous and fault-prone. Previously, grid resource What is grid computing? Grid computing is the ability to use and control networkconnected resources that are heterogeneous, distributed, managed independently and potentially faulty. Grid computing has received a lot of press recently with vendors such as IBM, Sun and HP each putting forth their grid strategies. One criticial aspect of grid computing, particularly for large-scale scientific applications, is resource management. Dispelling Seven Myths about Grid Resource Management 2 management meant finding CPU cycles for running long, compute-intensive, parallel jobs. However, grid resource management now means the manner in which resources are allocated, assigned, authenticated, authorised, assured, accessed, accounted and audited. We will contrast these two views of grid resource management in this article. In §1, we will present the beliefs of the previous view as myths because we find no basis for their continuance. We will also dispel these myths with counterexamples. In doing so, we will fashion the current view of grid resource management as a collection of observations that encompass the meaning of grid resource management and outline the tasks associated with it. When presenting this view, we will draw on the wealth of experience present in the grid community. Several grid projects share the observations we make here. We implemented the current view of grid resource management in Legion, a software infrastructure developed originally at the University of Virginia in the mid-1990s [GRIM97]. In 2001, a company called AVAKI Corporation was founded to provide commercial grid solutions to companies. The underlying architecture of Avaki 2.x is the same as that of Legion, and our discussion here applies to both. A key feature of Legion is its resource management framework. Grid resource management is a complex task, involving security and fault-tolerance as well as scheduling. Grid scheduling itself requires not a one-size-fits-all scheduler but an architectural framework that can accommodate different schedulers for different classes of problems. We will present the broad architectural features of the Legion resource management framework in §2. In §3, we will discuss some of the lessons we learnt in Legion regarding grid resource management. * The quote in the text seems to be the most popular variant. A variant with citation information is "A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable.", Leslie Lamport, as quoted in CACM, June 1992.

Grid resource allocation: allocation mechanisms and utilisation patterns

… of the sixth Australasian workshop on …, 2008

Grid systems have been put to remarkable use in re-cent years. Finding planets, rendering multi-million dollar movies, and helping to understand disease are just some of the examples grid systems have been used for. With business turning to towards using grid sys-tems ...