Spencer Smith | McMaster University (original) (raw)

Papers by Spencer Smith

ArXiv, 2018

Different communities rely heavily on software, but use quite different software development prac... more Different communities rely heavily on software, but use quite different software development practices. {\bf Objective}: We wanted to measure the state of the practice in the area of statistical software for psychology to understand how it compares to best practices. {\bf Method}: We compared and ranked 30 software tools with respect to adherence to best software engineering practices on items that could be measured by end-users. {\bf Results} We found that R packages use quite good practices, that while commercial packages were quite usable, many aspects of their development is too opaque to be measures, and that research projects vary a lot in their practices. {\bf Conclusion} We recommend that more organizations adopt practices similar to those used by CRAN to facilitate success, even for small teams. We also recommend close coupling of source code and documentation, to improve verifiability.

Advances in Engineering Software, 2016

We analyze the state of development practices for Mesh Generation and Mesh Processing (MGMP) soft... more We analyze the state of development practices for Mesh Generation and Mesh Processing (MGMP) software by comparing 27 MGMP projects. The analysis employs a reproducible method based on a grading template of 56 questions covering 13 software qualities. The software is ranked using the Analytic Hierarchy Process (AHP), a multicriteria decision making method appropriate for cases with a mix of qualitative and quantitative factors. The results reveal concerns regarding the maintainability, usability, reusability and performance of some MGMP software. Five recommendations are presented as feedback to the MGMP community: (i) Use an issue tracker for bug management and support requests. (ii) Document performance measures. (iii) Increase the use of libraries to promote software re-use to avoid "re-inventing the wheel." (iv) Improve reproducibility by recording the set up details for the development and testing environments. (v) Improve confidence in correctness through requirements specification, formal specification languages, and automated testing.

arXiv (Cornell University), Nov 26, 2019

We present GOOL, a Generic Object-Oriented Language. It demonstrates that a language, with the ri... more We present GOOL, a Generic Object-Oriented Language. It demonstrates that a language, with the right abstractions, can capture the essence of object-oriented programs. We show how GOOL programs can be used to generate humanreadable, documented and idiomatic source code in multiple languages. Moreover, in GOOL, it is possible to express common programming idioms and patterns, from simple library-level functions, to simple tasks (command-line arguments, list processing, printing), to more complex patterns, such as methods with a mixture of input, output and in-out parameters, and finally Design Patterns (such as Observer, State and Strategy). GOOL is an embedded DSL in Haskell that can generate code in Python, Java, C#, and C++.

arXiv (Cornell University), Dec 15, 2021

Our goal is to identify inhibitors and catalysts for productive longterm scientific software deve... more Our goal is to identify inhibitors and catalysts for productive longterm scientific software development. The inhibitors and catalysts could take the form of processes, tools, techniques, environmental factors (like working conditions) and software artifacts (such as user manuals, unit tests, design documents and code). The effort (time) invested in catalysts will pay off in the long-term, while inhibitors will take up resources, and can lower product quality. Developer surveys on inhibitors and catalysts will yield responses as varied as the education and experiential backgrounds of the respondents. Although well-meaning, responses will predictably be biased. For instance, developers may be guilty of the sunk cost fallacy, promoting a technology they have invested considerable hours in learning, even if the current costs outweigh the benefits. Likewise developers may recommend against spending time on proper requirements, not as an indication that requirements are not valuable, only that current practice doesn't promote requirements [2]. Another perceived inhibitor is time spent in meetings. For instance, the lack of visible short-term benefits renders department retreats unpopular, even though relationship building and strategic decision making may provide significant future rewards. Evaluating the usefulness of meetings is difficult. Rather than relying on preference and perception, as these examples illustrate, we need to measure the long-term impact of development choices to make wise ones.

arXiv (Cornell University), Oct 21, 2021

To improve software development methods and tools for research software, we first need to underst... more To improve software development methods and tools for research software, we first need to understand the current state of the practice. Therefore, we have developed a methodology for assessing the state of the software development practices for a given research software domain. The methodology is applied to one domain at a time in recognition that software development in different domains is likely to have adopted different best practices. Moreover, providing a means to measure different domains facilitates comparison of development practices between domains. For each domain we wish to answer questions such as: i) What artifacts (documents, code, test cases, etc.) are present? ii) What tools are used? iii) What principles, process and methodologies are used? iv) What are the pain points for developers? v) What actions are used to improve qualities like maintainability and reproducibility? To answer these questions, our methodology prescribes the following steps: i) Identify the domain; ii) Identify a list of candidate software packages; iii) Filter the list to a length of about 30 packages; iv) Gather source code and documentation for each package; v) Collect repository related data on each software package, like number of stars, number of open issues, number of lines of code; vi) Fill in the measurement template (the template consists of 108 questions to assess 9 qualities (including the qualities of installability, usability and visibility)); vii) Interview developers (the interview consists of 20 questions and takes about an hour); viii) Rank the software using the Analytic Hierarchy Process (AHP); and, ix) Analyze the data to answer the questions posed above. A domain expert should be engaged throughout the process, to ensure that implicit information about the domain is properly represented and to assist with conducting an analysis of the commonalities and variabilities between the 30 selected packages. Using our methodology, spreadsheet templates and AHP tool, we estimate (based on our experience with using the process) the time to complete an assessment for a given domain at 173 person hours.

The IFIP Working Group 2.5 on Numerical Software (IFIPWG2.5) wrote on 5th Septem- ber 2007 to the... more The IFIP Working Group 2.5 on Numerical Software (IFIPWG2.5) wrote on 5th Septem- ber 2007 to the IEEE Standards Committee concerned with revising the IEEE Floating- Point Arithmetic Standards 754 and 854 (IEEE754R), expressing the unanimous request of IFIPWG2.5 that the following requirement be included in the future computer arithmetic standard: For the data format double precision, interval arithmetic should be made available at the speed of simple floating-point arithmetic. IEEE754R (we believe) welcomed this development. They had before them a document defining interval arithmetic operations but, to be the basis of a standards document, it needed more detail. Members of the Interval Subroutine Library (ISL) team were asked to comment, in an email from Ulrich Kulisch that enclosed one from Jim Demmel to Van Snyder raising the issue. This paper provides ISL's comments.

Proceedings of the 2020 ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation, 2020

We present GOOL, a Generic Object-Oriented Language. GOOL shows that with the right abstractions,... more We present GOOL, a Generic Object-Oriented Language. GOOL shows that with the right abstractions, a language can capture the essence of object-oriented programs. GOOL generates human-readable, documented and idiomatic code in Python, Java, C#, and C++. In it, we can express common programming idioms and patterns. CCS Concepts • Software and its engineering → Source code generation; Abstraction, modeling and modularity; Object oriented languages.

We wish to understand the sustainability of current research software development practices. Sust... more We wish to understand the sustainability of current research software development practices. Sustainability is here defined to mean production of software that meets the (explicit and implicit) requirements of the present, while also allowing for cost-effective modifications in the future. To assess current sustainability, we will:<br> i) analyze existing open source projects to determine what documents, code and scripts are produced; ii) what concepts are captured by those artifacts; iii) analyze textbooks and other authoritative sources to determine what documents, code and scripts are recommended;iv) determine what concepts are supposed to be captured by those artifacts. We will conduct some case studies on specific domains (such as medical imaging software and Lattice Boltzmann Solvers) via ranking representative software in pair-wise comparisons along multiple quality measures. For the top software thus identified, we will conduct simple experiments to assess usability an...

Software documentation improves qualities such as maintainability, reusability, verifiability and... more Software documentation improves qualities such as maintainability, reusability, verifiability and usability. So why do scientific software developers underemphasize documentation? Reasons include: i) requirements are not known up-front (they emerge over time), ii) change is frequent, iii) rigid processes hamper creativity, iv) the software is too complex; and, v) there is no test oracle. These are reasons that documentation is challenging, not excuses for avoiding it entirely. Complexity and frequent change are not unique to scientific software. So how do other domains deal with these challenges – by applying Software Engineering (SE) principles, techniques and tools. Specific SE ideas helpful to scientific software include: faking a rational design process, documentation templates, abstraction, anticipation of change, generality, replicability, separation of concerns, information hiding, and tool support. Many developers are familiar with these ideas, but examples of adapting them ...

Software qualities, like maintainability, reproducibility and verifiability, often suffer for Sci... more Software qualities, like maintainability, reproducibility and verifiability, often suffer for Scientific Computing Software (SCS) because of inadequate documentation, traceability, change-enabling design, and testing. Software developers would like to spend more time on documentation, and other software activities, but time and resource pressures frequently make this an unrealistic goal. Ideally, developers should be able to create traceable documentation, design, code, build scripts and tests, without the drudgery of writing and maintaining them. A potential solution is to generate the documentation, code and tests automatically by using Domain Specific Languages (DSLs) over a base of scientific, computing and documentation knowledge. This is the approach that is proposed for a new scientific software development framework called Drasil. By having one source of knowledge, along with rules for transforming and consistency checking, the qualities of completeness, consistency and trac...

The data provides a summary of the state of development practice for Geographic Information Syste... more The data provides a summary of the state of development practice for Geographic Information Systems (GIS) software (as of August 2017). The summary is based on grading a set of 30 GIS products using a template of 56 questions based on 13 software qualities. The products range in scope and purpose from a complete desktop GIS systems, to stand-alone tools, to programming libraries/packages. The template used to grade the software is found in the TabularSummaries.zip file. Each quality is measured with a series of questions. For unambiguity the responses are quantified wherever possible (e.g.~yes/no answers). The goal is for measures that are visible, measurable and feasible in a short time with limited domain knowledge. Unlike a comprehensive software review, this template does not grade on functionality and features. Therefore, it is possible that a relatively featureless product can outscore a feature-rich product. A virtual machine is used to provide an optimal testing environments for each software product. During the process of grading the 30 software products, it is much easier to create a new virtual machine to test the software on, rather than using the host operating system and file system. The raw data obtained by measuring each software product is in SoftwareGrading-GIS.xlsx. Each line in this file corresponds to between 2 and 4 hours of measurement time by a software engineer. The results are summarized for each quality in the TabularSummaries.zip file, as a tex file and compiled pdf file.

WIT transactions on engineering sciences, 2003

Transportation Research Part A: Policy and Practice, 1996

This paper draws on data collected in March 1993 from the M6 motorway north of Birmingham, Englan... more This paper draws on data collected in March 1993 from the M6 motorway north of Birmingham, England, to discuss the speeddflow relationchtps used in cost benefit analysis. Piecewise linear functions have been fitted to the M6 data, and are then compared to the relationships in the U.K. Cost Benefit Analysis Manual (COBA9: U.K. Department of Transport, 1981) and the U.S. Highway Capacity Manual (HCM: U.S. Transpn Res Bd, 1994). The results provide clear support for the value of 1200 v/h/l used in COBA as the flow at which speeds start to decrease more rapidly, but in other respects the results dikr from values in both COBA and the HCM. In comparison with the observed data, COBA estimates: a lower speed at a flow of 1200 v/h/l; a steeper slope at higher flows; a higher capacity flow; and a lower speed at capacity. Equivalently. HCM estimates: a gentler slope at higher flows; a higher capacity flow; and a higher speed at capacity. Thus the high-flow data fall xmewhere between the COBA and HCM estimates.

Proceedings of the Canadian Engineering Education Association, 2010

The laboratory infrastructure was deployed in two stages with the goal of providing a user experi... more The laboratory infrastructure was deployed in two stages with the goal of providing a user experience that was indistinguishable from a traditional computing lab. The first stage was a one-to-one mapping between 56 thin clients and 56 Blade servers using the Teradici communication protocol. The second stage was a true virtualised platform with 56 thin clients and 3 Blade servers using the PCover-IP (PCoIP) communication protocol. We surveyed several laboratory sections on usability and user experience of a popular visual programming development environment and a solid modelling CAD package. Students were asked to compare their current "workstation" performance against previous performance and the performance of other computing facilities on campus. When conducting the survey, students were not aware whether the machine they were seated at was a stage-1 or stage-2 configuration. The results show the majority of users believe the stage-2 virtualised implementation is as good or better than stage-1 or other facilities on campus. We have also compared resource requirements for the traditional computer lab and the virtualised computer lab. Operational resources are reduced by several orders of magnitude and administration has become significantly more efficient. In the current third stage of deployment we will be allowing students to connect to the virtualised laboratory from outside the physical lab. Our paper will cover the motivation and benefits for developing this laboratory, the structure of our virtualised laboratory, the student survey results, and the calculations of operational savings using this new laboratory model. The implementation of a virtualised laboratory structure provides new flexibility on the accessibility to computing laboratories, which we believe will be of interest to all levels of engineering education.

Proceedings of the Canadian Engineering Education Association, 2011

This paper presents a document driven strategic planning process for academic units. Details are ... more This paper presents a document driven strategic planning process for academic units. Details are provided on the steps in the process that will lead, with a reasonable investment of time and effort, to a quality strategic plan document that incorporates input from all stakeholders. The resulting plan consists of a mission statement, vision statement, goals, objectives and implementation ideas. The specific organization of the elements of the plan and tables for systematically presenting the details on definitions, relationships between components, evaluation criteria and timelines are presented using a template. The proposal for the template approach is motivated by the success of this approach in software engineering practise. The template facilitates producing a document that is complete, consistent, understandable and maintainable. The process and the final product are illustrated using examples from Engineering 1 at McMaster University.

Proceedings of the Canadian Engineering Education Association, 2013

investigated the correlation of predictive indicators for a relationship to identify students-at-... more investigated the correlation of predictive indicators for a relationship to identify students-at-risk of failure. As data sets begin to include a large number of potential indicators, methods of visual inspection or empirical comparison can be flawed or inefficient. The authors have begun applying statistical data reduction methods for the selection of ideal predictive indicators of students-at-risk. Once the predictors are selected, a random subset are used for the supervised training of the machine learning methodology. The Support-Vector-Machine is a machine learning algorithm that offers advantages of small training sets and tolerance of noisy data. The focus of this paper is not the actual predictors, but instead the theory and methodology of using data reduction and machine learning for selection of the best predictors of the interested researchers own data set. Example results from the authors initial analyses will be presented. This paper will be of interest to researchers, instructors, and administrators seeking an algorithmic approach to analyzing the large volume of data available for each engineering cohort.

Lecture Notes in Computer Science

The same methodology is used to develop 3 different applications. We begin by using a very expres... more The same methodology is used to develop 3 different applications. We begin by using a very expressive, appropriate Domain Specific Language, to write down precise problem definitions, using their most natural formulation. Once defined, the problems form an implicit definition of a unique solution. From the problem statement, our model, we use mathematical transformations to make the problem simpler to solve computationally. We call this crucial step "model manipulation." With the model rephrased in more computational terms, we can also derive various quantities directly from this model, which greatly simplify traditional numeric solutions, our eventual goal. From all this data, we then use standard code generation and code transformation techniques to generate lower-level code to perform the final numerical steps. This methodology is very flexible, generates faster code, and generates code that would have been all but impossible for a human programmer to get correct. 1 Note that where we use "model," mathematicians would use "problem" instead.

20th Conference on Software Engineering Education & Training (CSEET'07), 2007

This paper reports on the activities and results from the 3rd International Symposium on Software... more This paper reports on the activities and results from the 3rd International Symposium on Software Engineering Course Projects (SWECP 2006), which was held on October 19, 2006 in Toronto, Canada as part of the IBM CASCON 2006 conference. The symposium explored how educators, students, and industry can work together to develop a more rewarding educational experience for all stakeholders involved in the context of senior design classes and team-based course projects. Several key topics were discussed at the symposium, including team dynamics, the choice of software development process, and the role of advanced software engineering tools in design project courses. SWECP 2006 was a sequel to the workshops that were held in 2002 and 2005.