Computation: The Micro and the Macro View

The following sections are included:

• PREFACE

• CONTENTS

Shannon's classical channel capacity formula constraints on the rate information can be conveyed in terms of the noise to signal powers ratio, becoming arbitrarily large for dimming noise. Quantum mechanics imposes more severe constraints, regardless of the existence of external noise. In this article we review the quantum limitations on the information transmission rate in the steady state regime as well as the linear bound proposed by Bekenstein ten years ago. As we shall see, the latter is a direct consequence of indistinguishability of information carriers (quanta) and, therefore, is a fundamental constraint on the rate in which information can ever be conveyed.

Algorithmic information content is equal to the size — in the number of bits — of the shortest program for a universal Turing machine which can reproduce (i.e., plot with the requisite accuracy) a state of a physical system. In contrast to the statistical Boltzmann-Gibbs-Shannon entropy, which measures ignorance, the algorithmic information content is a measure of the available information. It is defined without a recourse to probabilities and can be regarded as a measure of randomness of a definite microstate. I suggest that the physical entropy S — that is, the quantity which determines the amount of the work ΔW which can be extracted in the cyclic isothermal expansion process through the equation ΔW = k_BTΔS — is a sum of two contributions: (i) The missing information measured by the usual statistical entropy and (ii) the known randomness measured by the algorithmic information content. The sum of these two contributions is a “constant of motion” in the process of a dissipationless measurement on an equilibrium ensemble. This conservation under a measurement, which can be traced back to the noiseless coding theorem of Shannon, is necessary to rule out existence of a successful Maxwell's demon.

We present a quantitative assessment of the value of cooperation for solving constraint satisfaction problems through a series of experiments, as well as a general theory of cooperative problem solving. These experiments, using both hierarchical and non-hierarchical cooperation, clearly exhibit a universal improvement in performance that results from cooperation. We also show both theoretically and experimentally the super-linear speed-up that results from having a diverse collection of skills among the cooperating agents. Our results suggest an alternative methodology to existing techniques for solving constraint satisfaction problems in computer science and distributed artificial intelligence.

Despite serious concerns raised by the proven ability of computer viruses to spread between individual systems and establish themselves as a persistent infection in the computer population, there have been very few efforts to analyze their propagation theoretically. The strong analogy between biological viruses and their computational counterparts has motivated us to adapt the techniques of mathematical epidemiology to the study of computer virus propagation. In order to allow for the most general patterns of program sharing, we extend a standard epidemiological model by placing it on a directed graph and use a combination of analysis and simulation to study its behavior. We determine the conditions under which epidemics are likely to occur, and in cases where they do, we explore the dynamics of the expected number of infected individuals as a function of time. We conclude that an imperfect defense against computer viruses can still be highly effective in preventing their widespread proliferation, provided that the infection rate does not exceed a well-defined critical epidemic threshold.

We view computers and problems to be simulated on them as physical systems. We show how the concepts of dimension, topology, size, temperature, adiabatic systems and phase transitions can be valuable in general computations. A space-time picture of computation allows one to discuss the load balancing and decomposition of processes as a theory of interacting strings. The mathematics underlying these ideas allows a quantitative description of the performance of parallel machines for different problems. We also describe a theory of the architecture or overall structure of problems. This approach can be naturally applied to parallelizing compilers which can be thought as mapping temporal constructs in “sequential” languages into spatial parallelism. We consider that current languages (C, Fortran, ADA …) as “wrong” (incomplete) as they do not properly express the spatial and temporal structure of problems.