Volta: Developing Distributed Applications by Recompiling (original) (raw)
Related papers
MAGE: A Distributed Programming Model
2001
This thesis explores computation mobility. We view mobility, the selection of an execution environment, as an attribute of a computation. To capture this attribute, we introduce a novel programming abstraction, which we call a mobility attribute, that specifies where computations should occur within a distributed system. Programmers dynamically bind mobility attributes to program components, nonempty sets of functions and their state. Once bound to a component, mobility attributes apply prior to each execution of that component.
D: A Language Framework for Distributed Programming
1997
Two of the most important issues in distributed systems are the synchronization of concurrentthreads and the application-level data transfers between execution spaces. At the design level, addressingthese issues typically requires analyzing the components under a different perspective thanis required to analyze the functionality. Very often, it also involves analyzing several componentsat the same time, because of the way those two
Generation of distributed programs in their target execution environment
Proceedings. 15th IEEE International Workshop on Rapid System Prototyping, 2004.
This paper presents how we use Lf P, a formalbased, Object Oriented notation dedicated to the development of distributed application. The language comes with a development methodology which emphasizes the separation between the control aspect of the application, and the computational aspect. Specifications written in Lf P focus on the control part of the application which is known to be a difficult issue of distributed applications. The corresponding code is then automatically generated to implement the behavior in the target execution environment. This paper briefly presents the Lf P language, how we handle connection between computational and control aspects. We then describes a prototype implementation of the code generator and the associated runtime. Note: the work presented in this paper is being performed withing the MORSE project. MORSE is a French government founded research project (RNTL) with industrial partners (Sagem, Aonix) and academic partners (LIP6-Univ. P. & M. Curie, LaBRI-Univ. Bordeaux I).
Regis: a constructive development environment for distributed programs
Distributed Systems Engineering, 1994
Regis is a programming environment aimed at supporting the development and execution of distributed programs. It embodies a constructive approach to the development of programs based on separating program structure from communication and computation. The emphasis is on constructing programs from multiple parallel computational components which cooperate to achieve the overall goal. The environment is designed to easily accommodate multiple communication mechanisms and primitives. Both the computational and communication elements of Regis programs are programmed in the Object Oriented programming language C++. The elements are combined into distributed programs using the configuration language Darwin. The paper describes programming in Regis through a set of small example programs drawn from the implementation of an Active Badge system.
2005
In this paper we introduce Causeway, operating system support facilitating the development of metaapplications, like priority scheduling and performance debugging, that control and analyze the execution of distributed programs. Meta-applications use Causeway to inject and access metadata on application execution paths to implement their specific goals. Causeway has two components: (1) interfaces to inject and access metadata and (2) mechanisms to automate propagation of metadata. Using Causeway we could rapidly implement a distributed priority scheduling system where priority of a task is injected and propagated as metadata, and accessed to implement global priority scheduling. This required writing only about 150 lines of code on top of Causeway. With this system we obtained global priority scheduling on an implementation of the TPC-W benchmark.
rpcgen: The Simple Way to Develop Distributed Applications
Open Source For You Magazine, 2017
In distributed computing, the data and the services processing the data are placed in different computers connected by a network. This makes developing a distributed application a daunting task. This article presents a simple method for developing a distributed application using rpcgen. rpcgen: The Simple Way to Develop Distributed Applications M any scientific problems are solved using distributed systems, as shown in Figure 1. Basically, a distributed system is a collection of autonomous computers connected by a network that appears to the users of the system as a single computer. The message passing mechanism plays a major role in the success of distributed computing. For a distributed application, developers have to worry about details such as sockets, network byte order, host byte order, etc. In this situation, Remote Procedure Call (RPC) presents a simple mechanism for distributed computing without the need to include socket code in the distributed application.
Lazy Interworking of Compiled and Interpreted Code for Sandboxing and Distributed Systems
Implementation and Application of Functional Languages (IFL '19), September 25–27, 2019, Singapore, Singapore, 2019
More and more applications rely on the safe execution of code unknown at compile-time, for example in the implementation of web browsers and plugin systems. Furthermore, these applications usually require some form of communication between the added code and its embedder, and hence a communication channel must be set up in which values are serialized and deserialized. This paper shows that in a functional programming language we can solve these two problems at once, if we realize that the execution of extra code is nothing more than the deserialization of a value which happens to be a function. To demonstrate this, we describe the implementation of a serialization library for the language Clean, which internally uses an interpreter to evaluate added code in a separate, sandboxed environment. Remarkable is that despite the conceptual asymmetry between "host" and "interpreter", lazy interworking must be implemented in a highly symmetric fashion, much akin to distributed systems. The library interworks on a low level with the native Clean program, but has been implemented without any changes to the native runtime system. It can therefore easily be ported to other programming languages. We can use the same technique in the context of the web, where we want to be able to share possibly lazy values between a server and a client. In this case the interpreter runs in WebAssembly in the browser and communicates seamlessly with the server, written in Clean. We use this in the iTasks web framework to handle communication and offload computations to the client to reduce stress on the server-side. Previously, this framework cross-compiled the Clean source code to JavaScript and used JSON for communication. The interpreter has a more predictable and better performance, and integration is much simpler because it interworks on a lower level with the web server.
The Weaves Reconfigurable Programming Framework
Arxiv preprint cs/0210031, 2002
This research proposes a language independent intra-process framework for object based composition of unmodified code modules. Intuitively, the two major programming models-threads and processes-can be considered as extremes along a sharing axis. Multiple threads through a process share all global state, whereas instances of a process (or independent processes) share no global state. Weaves provide the generalized framework that allows arbitrary (selective) sharing of state between multiple control flows through a process. In the Weaves framework a single process has the same level of complexity as a workstation, with independent "sub-processes", state sharing and scheduling, all of which is achieved without requiring any modification to existing code bases. Furthermore, the framework allows dynamic instantiation of code modules and control flows through them. In effect, weaves create intra-process modules (similar to objects in OOP) from code written in any language. Applications can be built by instantiating Weaves to form Tapestries of dynamically interacting code. The Weaves paradigm allows objects to be arbitrarily shared-it is a true superset of both processes as well as threads, with code sharing and fast context switching time similar to threads. Weaves do not require any special support from either the language or application code-practically any code can be weaved. Weaves also include support for fast automatic checkpointing and recovery with no application support. This paper presents the elements of the Weaves framework and results from our implementation that works by reverse-analyzing source-code independent ELF object files. The current implementation has been validated over Sweep3D, a benchmark for 3D discrete ordinates neutron transport [Koch et al., 1992], and a user-level port of the Linux 2.4 family kernel TCP/IP protocol stack. Performance results show that the context switch overhead in the Weaves framework is almost identical to threads. 2 Introduction With the growing complexity of software code bases, application designers are increasingly moving towards component based models, where applications are built by compositional modeling. The success of the object oriented programming paradigm, with its support for encapsulation and compartmentalization of functionality, has played a major role in facilitating this mode of application development. However, to make effective use of the OO paradigm, codes need to be written in a language that supports OOP. While this is the approach of choice for the next generation of applications, we argue that it is infeasible for several areas. Areas such as scientific computing have vast repositories of legacy non-OO codes. Rewriting and validating these applications in an OO language is an enormous software engineering endeavor. Furthermore, with the development of increasingly sophisticated sensors, compute devices have begun interacting with the real world. This interaction presents a new dimension to application adaptivity. Given the vast stream of data that flows from sensor arrays, it may not be possible to apriori determine all the possible modes in which an application may interact with the environment. In this scenario, applications would need to evolve both in code and state to meet the needs of a dynamically changing environment, a notion we call reconfigurable programming. This proposal presents a new language independent intra-process framework called Weaves, for object based composition of unmodified code modules. In effect, weaves create intra-process modules (similar to objects in OOP) from code written in any language. Applications can be built by instantiating Weaves to form Tapestries of dynamically interacting code. The Weaves paradigm allows objects to be arbitrarily shared-it is a true superset of both processes as well as threads, with code sharing and fast context switching time similar to threads. The main feature of Weaves is that it does not require any special support from either the language or application code-practically any code can be weaved. Our current prototype implementation uses post-compiler analysis on ELF object files, supporting the vast majority of UNIX platforms and language compilers that use ELF format. The Weaves framework being very general, presents several new directions for developing highperformance codes. First, it enables run time composition of legacy applications, without requiring that any code be rewritten or even modified. Second, weaves enables run-time adaptivity. Run-time adaptation requires support for checkpointing and recovery and support for dynamic insertion and removal of code modules. Currently, the majority of applications implement their own user-level checkpointing, which adds a lot to the complexity and maintainability of these code bases. The Weaves framework includes transparent support for fast checkpointing and recovery with no application support. Our post-compiler analysis automatically determines the necessary state that needs to be saved and restored and presents a simple interface to this functionality. Finally, weaves support run-time flow migration, which enables automatic load-balancing of composed codes on parallel machines. The ability to migrate parts of an application as opposed to the whole gives finer grain control over load-balancing decisions. This paper is organized as follows. In section 2 we motivate the needs for a transparent component-based framework. Section 3 describes the elements of the Weaves framework, including support for tuple spaces, automatic checkpointing and recovery, and code migration. In section 4, we present implementation details and evaluation results from the current prototype. Section 5 presents ongoing work in categorizing adaptivity, augmenting our declarative tapestry configuration with a high-level functional composition specification language, and using Weaves to provide run-time systems support for migration of parallel communication codes in a computational grid environment.
DIPC : A Heterogeneous Distributed Programming System
2003
Distributed Inter-Process Communication (DIPC) is a software-only solution to enable people to build and program Multi-Computers. Among other things, DIPC provides the programmers with Transparent Distributed Shared Memory. DIPC is not concerned with the incompatible executable code problem. It also does not change user’s data, but DIPC has to understand data produced by its own peer processes in different machines. This paper briefly describes the efforts done to make it a heterogeneous system by enabling it to function under the Linux operating system running on both Intel x86 and Motorola 680x0 processors. Here, different parts of the same application could execute in machines with different architectures
lcc.NET: targeting the .NET Common Intermediate Language from Standard C
Software: Practice and Experience, 2004
The core of the Microsoft .NET platform includes a new virtual machine (VM), the Common Intermediate Language, also known as MSIL. Unlike most other VMs, including the Java VM, MSIL is specifically designed to support a wide range of languages. While it is designed primarily for type-safe, objectoriented languages, it also has facilities that support both low-level languages and very high-level languages. For example, it accommodates unsafe pointer arithmetic and tail calls. This paper describes the implementation of a MSIL back end for lcc, a retargetable compiler for Standard C. C is at one end of the range of languages that MSIL intends to support and lcc is just about the simplest 'real' C compiler that is widely available. Porting lcc to MSIL thus provides a realistic test of how well MSIL supports this class of languages and provides a glimpse at its performance costs. This effort succeeded, but static initializations, function pointers, separate compilation and address arithmetic were major problem areas. These problems also suggested improvements to lcc's code-generation interface and they exposed a long-standing error in the lcc front end. Preliminary measurements suggest that programs compiled by the MSIL back end run two to three times slower than those compiled by lcc native Intel x86 back end, but the MSIL programs have some important diagnostic benefits.