Frank Löffler | Louisiana State University (original) (raw)

Papers by Frank Löffler

Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment on Bridging from the eXtreme to the campus and beyond - XSEDE '12, 2012

ABSTRACT An executable paper is a hypermedia for publishing, reviewing, and reading scholarly pap... more ABSTRACT An executable paper is a hypermedia for publishing, reviewing, and reading scholarly papers which include a complete HPC software development or scientific code. A hypermedia is an integrated interface to multimedia including text, figures, video, and executables, on a subject of interest. Results within the executable paper include numeric output, graphs, charts, tables, equations and the underlying codes which generated such results. These results are dynamically regenerated and included in the paper upon recompilation and re-execution of the code. This enables a scientifically enriched environment which functions not only as a journal but as a laboratory in itself, in which readers and reviewers may interact with and validate the results. The Prickly Pear Archive (PPA) is such a system [2]. One distinguishing feature of the PPA is the inclusion of an underlying component-based simulation framework, Cactus [8], which simplifies the process of composing, compiling, and executing simulation codes. Code creation is simplified using common bits of infrastructure; each paper augments to the functionality of the framework. New distinguishing features include the (1) portability and (2) reproducibility of the archive, which allow researchers to move and re-create the software environment in which the simulation code was created. Further, the (3) Piraha parser is now used to match complex multi-line expressions inside parameter and LaTEX files. Finally, (4) an altogether new web interface has been created. The new interface options closely mirror the directory structure within the paper itself, which gives the reader a transparent view of the paper. Thus, once accustomed to reading from the archive, assembling a paper package becomes a straightforward and intuitive process. A PPA production system hosted on HPC resources (e.g. an XSEDE machine) unifies the computational scientific process with the publication process. A researcher may use the production archive to test simulations; and upon arriving at a scientifically meaningful result, the user may then incorporate the result in an executable paper on the very same resource the simulation was conducted. Housed within a virtual machine, the PPA allows multiple accounts within the same production archive, enabling users across campuses to bridge their efforts in developing scientific codes.

2013 International Conference on Social Computing, 2013

ABSTRACT This paper details Inkling, a generalized executable paper system for generating hyperme... more ABSTRACT This paper details Inkling, a generalized executable paper system for generating hypermedia. Whereas a traditional paper has static content derived from the data, i.e. tables, charts, graphs, and animations, the executable paper dynamically generates these using an underlying code and editable input parameters specified in the paper itself. By use of a language which may be seamlessly incorporated into the paper text and made transparent to the reader or reviewer, the system allows for ease of both use and validation. Novel in our system is (1)generality, in that it provides a generic coupling between the paper-generating infrastructure and the backend science code, (2) a minimalist text-based human-readable input format which abstracts algorithms from the reader and reviewer, (3) out-of-order dependency-based execution, which allows the author to chain outputs to inputs, and (4) a scheme for building a database of author-contributed codes which may be easily shared, reused and referenced.

International Journal of Modern Physics A, 2013

We give an introduction to the Einstein Toolkit, a mature, open-source computational infrastructu... more We give an introduction to the Einstein Toolkit, a mature, open-source computational infrastructure for numerical relativity based on the Cactus Framework, for the target group of new users. This toolkit is composed of several different modules, is developed by researchers from different institutions throughout the world and is in active continuous development. Documentation for the toolkit and its several modules is often scattered across different locations, a difficulty new users may at times have to struggle with. Scientific papers exist describing the toolkit and its methods in detail, but they might be overwhelming at first. With these lecture notes we hope to provide an initial overview for new users. We cover how to obtain, compile and run the toolkit, and give an overview of some of the tools and modules provided with it.

Physical Review D, 2015

We present results on the effect of the stiffness of the equation of state on the dynamical barmo... more We present results on the effect of the stiffness of the equation of state on the dynamical barmode instability in rapidly rotating polytropic models of neutron stars in full General Relativity. We determine the change in the threshold for the emergence of the instability for a range of the adiabatic Γ index from 2.0 to 3.0, including two values chosen to mimic more realistic equations of state at high densities. PACS numbers: 04.25.D-, 04.40.Dg, 95.30.Lz, 97.60.Jd

Scientific Programming, 2013

Starting from a high-level problem description in terms of partial differential equations using a... more Starting from a high-level problem description in terms of partial differential equations using abstract tensor notation, the Chemora framework discretizes, optimizes, and generates complete high performance codes for a wide range of compute architectures. Chemora extends the capabilities of Cactus, facilitating the usage of large-scale CPU/GPU systems in an efficient manner for complex applications, without low-level code tuning. Chemora achieves parallelism through MPI and multi-threading, combining OpenMP and CUDA. Optimizations include high-level code transformations, efficient loop traversal strategies, dynamically selected data and instruction cache usage strategies, and JIT compilation of GPU code tailored to the problem characteristics. The discretization is based on higher-order finite differences on multi-block domains. Chemora's capabilities are demonstrated by simulations of black hole collisions. This problem provides an acid test of the framework, as the Einstein equations contain

The advent of the Petascale era provides a great opportunity as well as a great challenge for com... more The advent of the Petascale era provides a great opportunity as well as a great challenge for computational science and engineering. In order to fully leverage the resources available, scientific applications need to scale to unprecedented numbers of processing cores and adapt to multicore architectures with complex memory and network hierarchies. In the numerical relativity community, the XiRel project has been funded by NSF to help prepare for these upcoming resources. The central goal of XiRel is to develop a highly scalable, efficient computational infrastructure based on the Carpet adaptive mesh refinement library that is fully integrated into the Cactus framework and optimized for numerical relativity applications. This paper presents our work towards building and benchmarking such an infrastructure which will benefit a wide spectrum of scientific applications that are based on Cactus.

The Cactus Framework is an open-source, modular, portable programming environment for the collabo... more The Cactus Framework is an open-source, modular, portable programming environment for the collaborative development and deployment of scientific applications using high-performance computing. Its roots reach back to 1996 at the National Center for Supercomputer Applications and the Albert Einstein Institute in Germany, where its development jumpstarted. Since then, the Cactus framework has witnessed major changes in hardware infrastructure as well as its own community. This paper describes its endurance through these past changes and, drawing upon lessons from its past, also discusses future challenges for Cactus.

ABSTRACT Large, complex, multi-scale, multi-physics simulation codes, running on high performance... more ABSTRACT Large, complex, multi-scale, multi-physics simulation codes, running on high performance com-puting (HPC) platforms, have become essential to advancing science and engineering. These codes simulate multi-scale, multi-physics phenomena with unprecedented fidelity on petascale platforms, and are used by large communities. Continued ability of these codes to run on future platforms is as crucial to their communities as continued improvements in instruments and facilities are to experimental scientists. However, the ability of code developers to do these things faces a serious challenge with the paradigm shift underway in platform architecture. The complexity and uncertainty of the future platforms makes it essential to approach this challenge cooperatively as a community. We need to develop common abstractions, frameworks, programming models and software development methodologies that can be applied across a broad range of complex simulation codes, and common software infrastructure to support them. In this position paper we express and discuss our belief that such an infrastructure is critical to the deployment of existing and new large, multi-scale, multi-physics codes on future HPC platforms.

Proceedings - IEEE International Conference on Cluster Computing, ICCC, 2009

This paper motivates how collaborative tools are needed to support modern computational science, ... more This paper motivates how collaborative tools are needed to support modern computational science, using the case study of a distributed team of numerical relativists developing and running codes to model black holes and other astrophysical objects. We describe a summary of previous tools developed within the collaboration, and how they are integrated and used with their simulation codes which are built using the Cactus Framework.

Proceedings of the 2010 TeraGrid Conference, TG '10, 2010

Assembling simulation software along with the associated tools and utilities is a challenging end... more Assembling simulation software along with the associated tools and utilities is a challenging endeavor, particularly when the components are distributed across multiple source code versioning systems. It is problematic for researchers compiling and running the software across many different supercomputers, as well as for novices in a field who are often presented with a bewildering list of software to collect and install.

Proceedings - IEEE/ACM International Workshop on Grid Computing, 2010

Component frameworks are complex systems that rely on many layers of abstraction to function prop... more Component frameworks are complex systems that rely on many layers of abstraction to function properly. One essential requirement is a consistent means of describing each individual component and how it relates to both other components and the whole framework. As component frameworks are designed to be flexible by nature, the description method should be simultaneously powerful, lead to efficient code, and be easy to use, so that new users can quickly adapt their own code to work with the framework.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2010

This paper is a report on experiences in benchmarking I/O performance on leading computational fa... more This paper is a report on experiences in benchmarking I/O performance on leading computational facilities on the NSF TeraGrid network with a large scale scientific application. Instead of focusing only on the raw file I/O bandwidth provided by different machine architectures, the I/O performance and scalability of the computational tools and libraries that are used in current production simulations are tested as a whole, however with focus mostly on bulk transfers. It is seen that the I/O performance of our production code scales very well, but is limited by the I/O system itself at some point. This limitation occurs at a low percentage of the computational size of the machines, which shows that at least for the application used for this paper the I/O system can be an important limiting factor in scaling up to the full size of the machine.

Proceedings of the TeraGrid 2011 Conference: Extreme Digital Discovery, TG'11, 2011

Abstract We describe how a new graduate course in scientific computing, taught during Fall 2010 a... more Abstract We describe how a new graduate course in scientific computing, taught during Fall 2010 at Louisiana State University, utilized TeraGrid resources to familiarize students with some of the real world issues that computational scientists regularly deal with in their work. The course was designed to provide a broad and practical introduction to scientific computing, creating the basic skills and experience to very quickly get involved in research projects involving modern cyberinfrastructure and complex real world scientific problems. ...

Proceedings of the TeraGrid 2011 Conference: Extreme Digital Discovery, TG'11, 2011

Abstract This paper describes the Alpaca runtime tools. These tools leverage the component infras... more Abstract This paper describes the Alpaca runtime tools. These tools leverage the component infrastructure of the Cactus Framework in a novel way to enable runtime steering, monitoring, and interactive control of a simulation. Simulation data can be observed graphically, or by inspecting values of variables. When GPUs are available, images can be generated using volume ray casting on the live data. In response to observed error conditions or automatic triggers, users can pause the simulation to modify or repair data, ...

We present a simple method for applying excision boundary conditions for the relativistic Euler e... more We present a simple method for applying excision boundary conditions for the relativistic Euler equations. This method depends on the use of Reconstruction-Evolution methods, a standard class of HRSC methods. We test three different reconstruction schemes, namely TVD, PPM and ENO. The method does not require that the coordinate system is adapted to the excision boundary. We demonstrate the effectiveness of our method using tests containing discontinuites, static test-fluid solutions with black holes, and full dynamical collapse of a neutron star to a black hole. A modified PPM scheme is introduced because of problems arisen when matching excision with the original PPM reconstruction scheme.

Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment on Bridging from the eXtreme to the campus and beyond - XSEDE '12, 2012

ABSTRACT An executable paper is a hypermedia for publishing, reviewing, and reading scholarly pap... more ABSTRACT An executable paper is a hypermedia for publishing, reviewing, and reading scholarly papers which include a complete HPC software development or scientific code. A hypermedia is an integrated interface to multimedia including text, figures, video, and executables, on a subject of interest. Results within the executable paper include numeric output, graphs, charts, tables, equations and the underlying codes which generated such results. These results are dynamically regenerated and included in the paper upon recompilation and re-execution of the code. This enables a scientifically enriched environment which functions not only as a journal but as a laboratory in itself, in which readers and reviewers may interact with and validate the results. The Prickly Pear Archive (PPA) is such a system [2]. One distinguishing feature of the PPA is the inclusion of an underlying component-based simulation framework, Cactus [8], which simplifies the process of composing, compiling, and executing simulation codes. Code creation is simplified using common bits of infrastructure; each paper augments to the functionality of the framework. New distinguishing features include the (1) portability and (2) reproducibility of the archive, which allow researchers to move and re-create the software environment in which the simulation code was created. Further, the (3) Piraha parser is now used to match complex multi-line expressions inside parameter and LaTEX files. Finally, (4) an altogether new web interface has been created. The new interface options closely mirror the directory structure within the paper itself, which gives the reader a transparent view of the paper. Thus, once accustomed to reading from the archive, assembling a paper package becomes a straightforward and intuitive process. A PPA production system hosted on HPC resources (e.g. an XSEDE machine) unifies the computational scientific process with the publication process. A researcher may use the production archive to test simulations; and upon arriving at a scientifically meaningful result, the user may then incorporate the result in an executable paper on the very same resource the simulation was conducted. Housed within a virtual machine, the PPA allows multiple accounts within the same production archive, enabling users across campuses to bridge their efforts in developing scientific codes.

2013 International Conference on Social Computing, 2013

ABSTRACT This paper details Inkling, a generalized executable paper system for generating hyperme... more ABSTRACT This paper details Inkling, a generalized executable paper system for generating hypermedia. Whereas a traditional paper has static content derived from the data, i.e. tables, charts, graphs, and animations, the executable paper dynamically generates these using an underlying code and editable input parameters specified in the paper itself. By use of a language which may be seamlessly incorporated into the paper text and made transparent to the reader or reviewer, the system allows for ease of both use and validation. Novel in our system is (1)generality, in that it provides a generic coupling between the paper-generating infrastructure and the backend science code, (2) a minimalist text-based human-readable input format which abstracts algorithms from the reader and reviewer, (3) out-of-order dependency-based execution, which allows the author to chain outputs to inputs, and (4) a scheme for building a database of author-contributed codes which may be easily shared, reused and referenced.

International Journal of Modern Physics A, 2013

We give an introduction to the Einstein Toolkit, a mature, open-source computational infrastructu... more We give an introduction to the Einstein Toolkit, a mature, open-source computational infrastructure for numerical relativity based on the Cactus Framework, for the target group of new users. This toolkit is composed of several different modules, is developed by researchers from different institutions throughout the world and is in active continuous development. Documentation for the toolkit and its several modules is often scattered across different locations, a difficulty new users may at times have to struggle with. Scientific papers exist describing the toolkit and its methods in detail, but they might be overwhelming at first. With these lecture notes we hope to provide an initial overview for new users. We cover how to obtain, compile and run the toolkit, and give an overview of some of the tools and modules provided with it.

Physical Review D, 2015

We present results on the effect of the stiffness of the equation of state on the dynamical barmo... more We present results on the effect of the stiffness of the equation of state on the dynamical barmode instability in rapidly rotating polytropic models of neutron stars in full General Relativity. We determine the change in the threshold for the emergence of the instability for a range of the adiabatic Γ index from 2.0 to 3.0, including two values chosen to mimic more realistic equations of state at high densities. PACS numbers: 04.25.D-, 04.40.Dg, 95.30.Lz, 97.60.Jd

Scientific Programming, 2013

Starting from a high-level problem description in terms of partial differential equations using a... more Starting from a high-level problem description in terms of partial differential equations using abstract tensor notation, the Chemora framework discretizes, optimizes, and generates complete high performance codes for a wide range of compute architectures. Chemora extends the capabilities of Cactus, facilitating the usage of large-scale CPU/GPU systems in an efficient manner for complex applications, without low-level code tuning. Chemora achieves parallelism through MPI and multi-threading, combining OpenMP and CUDA. Optimizations include high-level code transformations, efficient loop traversal strategies, dynamically selected data and instruction cache usage strategies, and JIT compilation of GPU code tailored to the problem characteristics. The discretization is based on higher-order finite differences on multi-block domains. Chemora's capabilities are demonstrated by simulations of black hole collisions. This problem provides an acid test of the framework, as the Einstein equations contain

The advent of the Petascale era provides a great opportunity as well as a great challenge for com... more The advent of the Petascale era provides a great opportunity as well as a great challenge for computational science and engineering. In order to fully leverage the resources available, scientific applications need to scale to unprecedented numbers of processing cores and adapt to multicore architectures with complex memory and network hierarchies. In the numerical relativity community, the XiRel project has been funded by NSF to help prepare for these upcoming resources. The central goal of XiRel is to develop a highly scalable, efficient computational infrastructure based on the Carpet adaptive mesh refinement library that is fully integrated into the Cactus framework and optimized for numerical relativity applications. This paper presents our work towards building and benchmarking such an infrastructure which will benefit a wide spectrum of scientific applications that are based on Cactus.

The Cactus Framework is an open-source, modular, portable programming environment for the collabo... more The Cactus Framework is an open-source, modular, portable programming environment for the collaborative development and deployment of scientific applications using high-performance computing. Its roots reach back to 1996 at the National Center for Supercomputer Applications and the Albert Einstein Institute in Germany, where its development jumpstarted. Since then, the Cactus framework has witnessed major changes in hardware infrastructure as well as its own community. This paper describes its endurance through these past changes and, drawing upon lessons from its past, also discusses future challenges for Cactus.

ABSTRACT Large, complex, multi-scale, multi-physics simulation codes, running on high performance... more ABSTRACT Large, complex, multi-scale, multi-physics simulation codes, running on high performance com-puting (HPC) platforms, have become essential to advancing science and engineering. These codes simulate multi-scale, multi-physics phenomena with unprecedented fidelity on petascale platforms, and are used by large communities. Continued ability of these codes to run on future platforms is as crucial to their communities as continued improvements in instruments and facilities are to experimental scientists. However, the ability of code developers to do these things faces a serious challenge with the paradigm shift underway in platform architecture. The complexity and uncertainty of the future platforms makes it essential to approach this challenge cooperatively as a community. We need to develop common abstractions, frameworks, programming models and software development methodologies that can be applied across a broad range of complex simulation codes, and common software infrastructure to support them. In this position paper we express and discuss our belief that such an infrastructure is critical to the deployment of existing and new large, multi-scale, multi-physics codes on future HPC platforms.

Proceedings - IEEE International Conference on Cluster Computing, ICCC, 2009

This paper motivates how collaborative tools are needed to support modern computational science, ... more This paper motivates how collaborative tools are needed to support modern computational science, using the case study of a distributed team of numerical relativists developing and running codes to model black holes and other astrophysical objects. We describe a summary of previous tools developed within the collaboration, and how they are integrated and used with their simulation codes which are built using the Cactus Framework.

Proceedings of the 2010 TeraGrid Conference, TG '10, 2010

Assembling simulation software along with the associated tools and utilities is a challenging end... more Assembling simulation software along with the associated tools and utilities is a challenging endeavor, particularly when the components are distributed across multiple source code versioning systems. It is problematic for researchers compiling and running the software across many different supercomputers, as well as for novices in a field who are often presented with a bewildering list of software to collect and install.

Proceedings - IEEE/ACM International Workshop on Grid Computing, 2010

Component frameworks are complex systems that rely on many layers of abstraction to function prop... more Component frameworks are complex systems that rely on many layers of abstraction to function properly. One essential requirement is a consistent means of describing each individual component and how it relates to both other components and the whole framework. As component frameworks are designed to be flexible by nature, the description method should be simultaneously powerful, lead to efficient code, and be easy to use, so that new users can quickly adapt their own code to work with the framework.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2010

This paper is a report on experiences in benchmarking I/O performance on leading computational fa... more This paper is a report on experiences in benchmarking I/O performance on leading computational facilities on the NSF TeraGrid network with a large scale scientific application. Instead of focusing only on the raw file I/O bandwidth provided by different machine architectures, the I/O performance and scalability of the computational tools and libraries that are used in current production simulations are tested as a whole, however with focus mostly on bulk transfers. It is seen that the I/O performance of our production code scales very well, but is limited by the I/O system itself at some point. This limitation occurs at a low percentage of the computational size of the machines, which shows that at least for the application used for this paper the I/O system can be an important limiting factor in scaling up to the full size of the machine.

Proceedings of the TeraGrid 2011 Conference: Extreme Digital Discovery, TG'11, 2011

Abstract We describe how a new graduate course in scientific computing, taught during Fall 2010 a... more Abstract We describe how a new graduate course in scientific computing, taught during Fall 2010 at Louisiana State University, utilized TeraGrid resources to familiarize students with some of the real world issues that computational scientists regularly deal with in their work. The course was designed to provide a broad and practical introduction to scientific computing, creating the basic skills and experience to very quickly get involved in research projects involving modern cyberinfrastructure and complex real world scientific problems. ...

Proceedings of the TeraGrid 2011 Conference: Extreme Digital Discovery, TG'11, 2011

Abstract This paper describes the Alpaca runtime tools. These tools leverage the component infras... more Abstract This paper describes the Alpaca runtime tools. These tools leverage the component infrastructure of the Cactus Framework in a novel way to enable runtime steering, monitoring, and interactive control of a simulation. Simulation data can be observed graphically, or by inspecting values of variables. When GPUs are available, images can be generated using volume ray casting on the live data. In response to observed error conditions or automatic triggers, users can pause the simulation to modify or repair data, ...

We present a simple method for applying excision boundary conditions for the relativistic Euler e... more We present a simple method for applying excision boundary conditions for the relativistic Euler equations. This method depends on the use of Reconstruction-Evolution methods, a standard class of HRSC methods. We test three different reconstruction schemes, namely TVD, PPM and ENO. The method does not require that the coordinate system is adapted to the excision boundary. We demonstrate the effectiveness of our method using tests containing discontinuites, static test-fluid solutions with black holes, and full dynamical collapse of a neutron star to a black hole. A modified PPM scheme is introduced because of problems arisen when matching excision with the original PPM reconstruction scheme.