Kevin Knuth | SUNY: University at Albany (original) (raw)

Papers by Kevin Knuth

Bulletin of the American Physical Society, 2016

Geodesics and Acceleration in Influence Theory JAMES WALSH, KEVIN KNUTH, University at Albany (SU... more Geodesics and Acceleration in Influence Theory JAMES WALSH, KEVIN KNUTH, University at Albany (SUNY) -Influence theory is concerned with a foundational approach where it is assumed that particles influence one another in a discrete one-to-one fashion. This results in a partially ordered set of influence events, called the influence network, where particles are represented by totally ordered chains of events. Information physics considers physical laws to result from consistent quantification of physical phenomena. Knuth and Bahreyni (2014) demonstrated that the mathematics of spacetime emerges from consistent quantification of influence events by embedded coordinated observers. Knuth (2014) showed that in 1+1 dimensions observer-based predictions about a free (uninfluenced) particle result in the Dirac equation. Here, we show that when a particle in 1+1 dimensions is influenced, it is uniquely and consistently described in terms of relativistic acceleration for constant rate of influence and in general obeys equations of the form of the geodesic equations of general relativity. This suggests that Influence Theory can also account for forces (like gravity), which give rise to well-known relativistic effects such as time dilation.

arXiv (Cornell University), Aug 15, 2013

I know about the universe because it influences me. Light excites the photoreceptors in my eyes, ... more I know about the universe because it influences me. Light excites the photoreceptors in my eyes, surfaces apply pressure to my touch receptors and my eardrums are buffeted by relentless waves of air molecules. My entire sensorium is excited by all that surrounds me. These experiences are all I have ever known, and for this reason, they comprise my reality. This essay considers a simple model of observers that are influenced by the world around them. Consistent quantification of information about such influences results in a great deal of familiar physics. The end result is a new perspective on relativistic quantum mechanics, which includes both a way of conceiving of spacetime as well as particle "properties" that may be amenable to a unification of quantum mechanics and gravity. Rather than thinking about the universe as a computer, perhaps it is more accurate to think about it as a network of influences where the laws of physics derive from both consistent descriptions and optimal informationbased inferences made by embedded observers. An Electron is an Electron because of What It Does As participants of the Information Age, we are all somewhat familiar with the electron. Currents of electrons flow through the wires of our devices bringing them power, transferring information and radiating signals through space. They tie us together enabling us to communicate with one another via the internet, as well as with distant robotic explorers on other worlds. Many of us feel like we have sensed electrons directly through the snap of an electric shock on a dry winter day or the flash and crash of a lightning bolt in a stormy summer sky. Electrons are bright, crackly sorts of things that jump and move unexpectedly from object to object. Yet they behave very predictably when confined to the wires of our electronic devices. But what are they really? Imagine that electrons could be pink and fuzzy. However, if each of these properties did not affect how an electron influences us or our measurement devices, then we would have no way of knowing about their pinkness or fuzziness. That is, if the fact that an electron was pink did not affect how it influenced others, then we would never be able to determine that electrons were pink. Knowledge about any property that does not affect how an electron exerts influence is inaccessible to us. We can turn this thought on its side. The only properties of an electron that we can ever know about are the ones that affect how an electron exerts influence. Another way to think about this is that an electron does not do what it does because it is an electron; rather an electron is an electron because of what it does. The conclusion is that the only properties of an electron that we can know about must be sufficiently describable in terms of how an electron influences others. That is, rather than imagining electrons to have properties such as position, speed, mass, energy, and so on, we are led to wonder if it might be possible, and perhaps better, to describe these attributes in terms of the way in which an electron influences. Since we cannot know what an electron is, perhaps it is best to simply focus on what an electron does.

arXiv (Cornell University), Nov 12, 2013

Source separation problems are ubiquitous in the physical sciences; any situation where signals a... more Source separation problems are ubiquitous in the physical sciences; any situation where signals are superimposed calls for source separation to estimate the original signals. In this tutorial I will discuss the Bayesian approach to the source separation problem. This approach has a specific advantage in that it requires the designer to explicitly describe the signal model in addition to any other information or assumptions that go into the problem description. This leads naturally to the idea of informed source separation, where the algorithm design incorporates relevant information about the specific problem. This approach promises to enable researchers to design their own high-quality algorithms that are specifically tailored to the problem at hand.

Philosophical Transactions of the Royal Society A, Nov 3, 2003

The last century saw the application of Boolean algebra to the construction of computing machines... more The last century saw the application of Boolean algebra to the construction of computing machines, which work by applying logical transformations to information contained in their memory. The development of information theory and the generalization of Boolean algebra to Bayesian inference have enabled these computing machines, in the last quarter of the twentieth century, to be endowed with the ability to learn by making inferences from data. This revolution is just beginning as new computational techniques continue to make difficult problems more accessible. Recent advances in our understanding of the foundations of probability theory have revealed implications for areas other than logic. Of relevance to intelligent machines, we recently identified the algebra of questions as the free distributive algebra, which will now allow us to work with questions in a way analogous to that which Boolean algebra enables us to work with logical statements. In this paper, we examine the foundations of inference and inquiry. We begin with a history of inferential reasoning, highlighting key concepts that have led to the automation of inference in modern machine-learning systems. We then discuss the foundations of inference in more detail using a modern viewpoint that relies on the mathematics of partially ordered sets and the scaffolding of lattice theory. This new viewpoint allows us to develop the logic of inquiry and introduce a measure describing the relevance of a proposed question to an unresolved issue. Last, we will demonstrate the automation of inference, and discuss how this new logic of inquiry will enable intelligent machines to ask questions. Automation of both inference and inquiry promises to allow robots to perform science in the far reaches of our solar system and in other star systems by enabling them not only to make inferences from data, but also to decide which question to ask, which experiment to perform, or which measurement to take given what they have learned and what they are designed to understand.

Digital Signal Processing, Dec 1, 2019

Histograms are convenient non-parametric density estimators, which continue to be used ubiquitous... more Histograms are convenient non-parametric density estimators, which continue to be used ubiquitously. Summary quantities estimated from histogram-based probability density models depend on the choice of the number of bins. We introduce a straightforward data-based method of determining the optimal number of bins in a uniform bin-width histogram. By assigning a multinomial likelihood and a noninformative prior, we derive the posterior probability for the number of bins in a piecewise-constant density model given the data. In addition, we estimate the mean and standard deviations of the resulting bin heights, examine the effects of small sample sizes and digitized data, and demonstrate the application to multi-dimensional histograms.

Nucleation and Atmospheric Aerosols, 2002

In celebration of the work of Richard Threlkeld Cox, we explore inductive logic and its role in s... more In celebration of the work of Richard Threlkeld Cox, we explore inductive logic and its role in science touching on both experimental design and analysis of experimental results. In this exploration we demonstrate that the duality between the logic of assertions and the logic of questions has important consequences. We discuss the conjecture that the relevance or bearing, b, of a question on an issue can be expressed in terms of the probabilities, p, of the assertions that answer the question via the entropy. In its application to the scientific method, the logic of questions, inductive inquiry, can be applied to design an experiment that most effectively addresses a scientific issue. This is performed by maximizing the relevance of the experimental question to the scientific issue to be resolved. It is shown that these results are related to the mutual information between the experiment and the scientific issue, and that experimental design is akin to designing a communication channel that most efficiently communicates information relevant to the scientific issue to the experimenter. Application of the logic of assertions, inductive inference (Bayesian inference) completes the experimental process by allowing the researcher to make inferences based on the information obtained from the experiment.

Entropy, Apr 4, 2014

The editors were recently made aware that a paper published in Entropy in 2003 [1] exhibited char... more The editors were recently made aware that a paper published in Entropy in 2003 [1] exhibited characteristics of duplication and self-plagiarism. After investigating the matter, and discussing the situation with the authors, they have offered to retract this paper. In particular, authors reused parts of their previous publications that appeared in Open Systems &

Journal of the Acoustical Society of America, Nov 1, 2006

Advances in Bayesian computational technology in the last decade have enabled the development of ... more Advances in Bayesian computational technology in the last decade have enabled the development of new source separation and source localization algorithms. These algorithms are greatly improved by the encoding of prior information about a specific problem in the form of the chosen relevant model parameters, the assignment of the likelihood functions, and the assignment of the prior probabilities of the model parameter values. I refer to such source separation algorithms as informed source separation for the reason that they are endowed with specific and often vital information about the problem. Furthermore, the Bayesian methodology allows source separation to be united with source localization simply by including the model parameters that are of interest to the researcher. Here, I will discuss the union of source separation and source localization under the Bayesian methodology, the incorporation of prior information, and the construction of an informed algorithm using the new computational technologies that allow us to estimate the values of the parameters that define these high-dimensional problems.

Nucleation and Atmospheric Aerosols, 2004

The Boolean lattice of logical statements induces the free distributive lattice of questions. Inc... more The Boolean lattice of logical statements induces the free distributive lattice of questions. Inclusion on this lattice is based on whether one question answers another. Generalizing the zeta function of the question lattice leads to a valuation called relevance or bearing, which is a measure of the degree to which one question answers another. Richard Cox conjectured that this degree can be expressed as a generalized entropy. With the assistance of yet another important result from Janos Aczél, I show that this is indeed the case, and that the resulting inquiry calculus is a natural generalization of information theory. This approach provides a new perspective on the Principle of Maximum Entropy. "A wise man's question contains half the answer." Solomon Ibn Gabirol (1021-1058) QUESTIONS AND ANSWERS Questions and answers, the unknown and the known, empty and full are all examples of duality. In this paper, I will show that a precise understanding of the duality of questions and answers allows one to determine the unique functional form of the relevance measure on questions that is consistent with the probability measure on the set of logical statements that form their answers. Much of the material presented in this paper relies on fundamental background material that I regrettably cannot take the space to address. While I provide a brief background below, I recommend the following previous papers [1, 2, 3, 4] in which more background, along with useful references, can be found. LATTICES AND VALUATIONS A partially ordered set, or poset for short, is a set of elements ordered according to a binary ordering relation, generically written ≤. One element b is said to 'include' another element a when a ≤ b. Inclusion on the poset is encoded by the zeta function

The frontiers collection, Dec 11, 2016

Well over a century after the discovery of the electron, we are still faced with serious conceptu... more Well over a century after the discovery of the electron, we are still faced with serious conceptual issues regarding precisely what an electron is. Since the development of particle physics and the Standard Model, we have accumulated a great deal of knowledge about the relationships among various subatomic particles. However, this knowledge has not significantly aided in our understanding of the fundamental nature of any particular elementary subatomic particle. The fact that many particle properties, such as position, time, speed, energy, momentum, and component of spin, are observerdependent suggests that these relevant variables do not represent properties per se, but rather the relationship between the observer and the observed. That is, they reflect details about how the electron influences the observer, and vice versa. Here we attempt to understand this by considering a simple model where particles influence one another in a discrete and direct fashion. The resulting framework, referred to as Influence Theory, is shown to faithfully reproduce a surprising amount of physics. While it would be naive to assume that the ideas presented here comprise anything resembling the final word on the matter, it is hoped that this work will demonstrate that a simple and understandable picture of particles, such as the electron, is indeed feasible and should be actively sought after.

arXiv (Cornell University), May 24, 2002

The problem of mixed signals occurs in many different contexts; one of the most familiar being ac... more The problem of mixed signals occurs in many different contexts; one of the most familiar being acoustics. The forward problem in acoustics consists of finding the sound pressure levels at various detectors resulting from sound signals emanating from the active acoustic sources. The inverse problem consists of using the sound recorded by the detectors to separate the signals and recover the original source waveforms. In general, the inverse problem is unsolvable without additional information. This general problem is called source separation, and several techniques have been developed that utilize maximum entropy, minimum mutual information, and maximum likelihood. In previous work, it has been demonstrated that these techniques can be recast in a Bayesian framework. This paper demonstrates the power of the Bayesian approach, which provides a natural means for incorporating prior information into a source model. An algorithm is developed that utilizes information regarding both the statistics of the amplitudes of the signals emitted by the sources and the relative locations of the detectors. Using this prior information, the algorithm finds the most probable source behavior and configuration. Thus, the inverse problem can be solved by simultaneously performing source separation and localization. It should be noted that this algorithm is not designed to account for delay times that are often important in acoustic source separation. However, a possible application of this algorithm is in the separation of electrophysiological signals obtained using electroencephalography (EEG) and magnetoencephalography (MEG).

arXiv (Cornell University), Apr 25, 2002

We examine the relationship between the Bayesian and information-theoretic formulations of source... more We examine the relationship between the Bayesian and information-theoretic formulations of source separation algorithms. This work makes use of the relationship between the work of Claude E. Shannon and the "Recent Contributions" by Warren Weaver (Shannon & Weaver 1949) as clarified by Richard T. Cox (1979) and expounded upon by Robert L. Fry (1996) as a duality between a logic of assertions and a logic of questions. Working with the logic of assertions requires the use of probability as a measure of degree of implication. This leads to a Bayesian formulation of the problem. Whereas, working with the logic of questions requires the use of entropy as a measure of the bearing of a question on an issue leading to an information-theoretic formulation of the problem.

The last century saw the application of Boolean algebra toward the construction of computing mach... more The last century saw the application of Boolean algebra toward the construction of computing machines, which work by applying logical transformations to information contained in their memory. The development of information theory and the generalization of Boolean algebra to Bayesian inference have enabled these computing machines. in the last quarter of the twentieth century, to be endowed with the ability to learn by making inferences from data. This revolution is just beginning as new computational techniques continue to make difficult problems more accessible. Xowever, modern intelligent machines work by inferring 'knowledge using oniy their pre-programmed prior knowledge and the data provided. They lack the ability to ask questions. or request data that would aid their inferences. Recent advances in understanding the foundations of probability theory have revealed implications for areas other than logic. Of relevance to intelligent machines, we identified the algebra of questions as the free distributive algebra, which now allows us to work with questions in a way analogous to that which Boolean algebra enables us to work with logical statements. In this paper we describe this logic of inference and inquiry using the mathematics of partially ordered sets and the scaffolding of lattice theory. discuss the far-reaching implications of the methodology, and demonstrate its application with current examples in machine learning. Automation of both inference and inquiry promises to allow robots to perform science in the far reaches of our solar system and in other star systems by enabling them to not only make inferences from data, but also decide which question to ask, experiment to perform, or measurement to take given what they have learned and what they are designed to understand.

International Conference on Artificial Intelligence and Statistics, 2005

For over a century, the study of logic has focused on the algebra of logical statements. This wor... more For over a century, the study of logic has focused on the algebra of logical statements. This work, first performed by George Boole, has led to the development of modern computers, and was shown by Richard T. Cox to be the foundation of Bayesian inference. Meanwhile the logic of questions has been much neglected. For our computing machines to be truly intelligent, they need to be able to ask relevant questions. In this paper I will show how the Boolean lattice of logical statements gives rise to the free distributive lattice of questions thus defining their algebra. Furthermore, there exists a quantity analogous to probability, called relevance, which quantifies the degree to which one question answers another. I will show that relevance is not only a natural generalization of information theory, but also forms its foundation.

arXiv (Cornell University), May 23, 2010

We present a derivation of special relativity based on the quantification of causally-ordered eve... more We present a derivation of special relativity based on the quantification of causally-ordered events. We postulate that events are fundamental, and that some events have the potential to influence other events, but not vice versa. This leads to the concept of a partially-ordered set of events, which is called a causal set. Quantification proceeds by selecting two chains of coordinated events, each of which represents an observer, and assigning a valuation to each chain. An event can be projected onto each chain by identifying the earliest event on the chain that can be informed about the event. In this way, events can be quantified by a pair of numbers, referred to as a pair, that derives from the valuations on the chains. Pairs can be decomposed into a sum of symmetric and antisymmetric pairs, which correspond to time-like and space-like coordinates. From this pair, we derive a scalar measure and show that this is the Minkowski metric. The Lorentz transformations follow, as well as the fact that speed is a relevant quantity relating two inertial frames, and that there exists a maximal speed, which is invariant in all inertial frames. Furthermore, the form of the Lorentz transformation in this picture offers a glimpse into the origin of spin. All results follow directly from the event postulate and the adopted quantification scheme.

Springer eBooks, 1998

High temporal resolution measurements of human brain activity can be performed by recording the e... more High temporal resolution measurements of human brain activity can be performed by recording the electric potentials on the scalp surface (electroencephalography, EEG), or by recording the magnetic fields near the surface of the head (magnetoencephalography, MEG). The analysis of the data is problematic due to the fact that multiple neural generators may be simultaneously active and the potentials and magnetic fields from these sources are superimposed on the detectors. It is highly desirable to un-mix the data into signals representing the behaviors of the original individual generators. This general problem is called blind source separation and several recent techniques utilizing maximum entropy, minimum mutual information, and maximum likelihood estimation have been applied. These techniques have had much success in separating signals such as natural sounds or speech, but appear to be ineffective when applied to EEG or MEG signals. Many of these techniques implicitly assume that the source distributions have a large kurtosis, whereas an analysis of EEG/MEG signals reveals that the distributions are multimodal. This suggests that more effective separation techniques could be designed for EEG and MEG signals.

arXiv (Cornell University), 1999

The problem of source separation is by its very nature an inductive inference problem. There is n... more The problem of source separation is by its very nature an inductive inference problem. There is not enough information to deduce the solution, so one must use any available information to infer the most probable solution. We demonstrate that source separation problems are well-suited for the Bayesian approach which provides a natural and logically consistent method by which one can incorporate prior knowledge to estimate the most probable solution given that knowledge. We derive the Bell-Sejnowski ICA algorithm from first principles, i.e. Bayes' Theorem and demonstrate how the Bayesian methodology makes explicit the underlying assumptions. We then further demonstrate the power of the Bayesian approach by deriving two separation algorithms that incorporate additional prior information. One algorithm separates signals that are known a priori to be decorrelated and the other utilizes information about the signal propagation through the medium from the sources to the detectors.

arXiv (Cornell University), Jan 21, 2015

We consider two areas of research that have been developing in parallel over the last decade: bli... more We consider two areas of research that have been developing in parallel over the last decade: blind source separation (BSS) and electromagnetic source estimation (ESE). BSS deals with the recovery of source signals when only mixtures of signals can be obtained from an array of detectors and the only prior knowledge consists of some information about the nature of the source signals. On the other hand, ESE utilizes knowledge of the electromagnetic forward problem to assign source signals to their respective generators, while information about the signals themselves is typically ignored. We demonstrate that these two techniques can be derived from the same starting point using the Bayesian formalism. This suggests a means by which new algorithms can be developed that utilize as much relevant information as possible. We also briefly mention some preliminary work that supports the value of integrating information used by these two techniques and review the kinds of information that may be useful in addressing the ESE problem.

arXiv: Instrumentation and Methods for Astrophysics, 2019

The search for life in the universe is a major theme of astronomy and astrophysics for the next d... more The search for life in the universe is a major theme of astronomy and astrophysics for the next decade. Searches for technosignatures are complementary to searches for biosignatures, in that they offer an alternative path to discovery, and address the question of whether complex (i.e. technological) life exists elsewhere in the Galaxy. This approach has been endorsed in prior Decadal Reviews and National Academies reports, and yet the field still receives almost no federal support in the US. Because of this lack of support, searches for technosignatures, precisely the part of the search of greatest public interest, suffers from a very small pool of trained practitioners. A major source of this issue is institutional inertia at NASA, which avoids the topic as a result of decades-past political grandstanding, conflation of the effort with non-scientific topics such as UFOs, and confusion regarding the scope of the term "SETI." The Astro2020 Decadal should address this issue ...