Algorithmic Reducibility

By: Bruce
January 13, 2011

…the chameleonic nature of numbers [is] so rich and complex that numerical patterns have the flexibility to mirror any other kind of pattern. (Douglas Hoftsadter in I am a Strange Loop, p. 159)

In my last post, I discussed the point of view known as ‘reductionism’ and the problems with that point of view. In summary, reductionism is the false belief that sciences that work with the smallest units of nature – atoms and below – are somehow more fundamental explanations of reality than emergent ones, such as thought or computation.

A few posts ago, I discussed computability and comprehension. My final conclusion was that while algorithms and explanations aren’t the same thing, you can’t have an explanation without having an algorithm.

Gregory Chaitin (in this article) points out that a theory must be simpler than the data it explains:

…a theory has to be simpler than the data it explains, otherwise it does not explain anything. The concept of a law becomes vacuous if arbitrarily high mathematical complexity is permitted, because then one can always construct a law no matter how random and patternless the data really are. (From “The Limits of Reason”)

Interestingly, this ability to reduce all explanations to computable algorithms forms a sort of ‘algorithmic reducibility’ that stands in stark contrast to the more familiar sort of ‘physical reducibility’ we normally think of.  In fact, if it’s true that all explanations have attached algorithms, then ‘algorithmic reducibility’ would seem to play the very role that Reductionists thought particle physics played: if you can’t reduce it to an algorithm, you don’t actually have a full explanation. Therefore this would mean that the theory of computation is actually more fundamental than particle physics.

There is something both disturbing and satisfying about the possibility that ‘algorithmic reducibility’ is fundamental. But it does seem to logically follow from our discussions so far. It also gives us a plausible explanation for why something like ‘beauty’ seems to be non-algorithmic but also doesn’t seem to be wholly subjective either. Perhaps we just don’t comprehend it yet. There are many phenomenon that we can’t yet turn into algorithms. We will always have to ask the question: does this mean I just don’t understand it yet, or does this phenomenon only exist in my mind?

Consciousness and Algorithms

But then what about consciousness? Is not consciousness effectively an aspect of nature too? Or is it separate from nature? Does this not lead to a conundrum? If consciousness can be explained, then it can be reduced to an algorithm. Doesn’t that mean we’re all just automatons? But if we say consciousness is not algorithmic, then doesn’t that mean we are claiming consciousness is fundamentally unexplainable? Aren’t we then claiming that not even God can comprehend consciousness because it’s beyond any sort of comprehension?

This paradox is a key question of interest for me and in future posts I will explore this further, along with some possible ways to solve this paradox. But first, we need to finish determining what science really is and come up with a good theory for how we gain knowledge. For the moment, we’ll temporarily declare consciousness off limits for algorithms and then re-evaluate the question when we are ready for it.

Questions for Discussion:

  1. Which is more disturbing to you? The possibility that we are algorithms, or the possibility that science can never, even in principle, explain consciousness?
  2. If science can never explain consciousness, then does that imply that even God can’t explain consciousness? If not, then does that not imply that ‘explanations” must not require algorithms? Then what is an explanation?

Tags: , , ,

17 Responses to Algorithmic Reducibility

  1. » Algorithmic Reducibility The Millennial Star on January 13, 2011 at 5:53 PM

    [...] Don’t miss the latest post in my “Reason as a Guide to Reality” series. [...]

    Like this comment? Thumb up 0

  2. Martin on January 13, 2011 at 6:25 PM

    I think you’d have to make a much stronger argument that consciousness couldn’t be reduced to an algorithm. Why couldn’t one model consciousness with an intricately nested hierarchy of values/objectives/rules and a random number generator? Many things that can’t be modeled at the micro-perspective can be well-modeled statistically at the macro-perspective. And we wouldn’t claim consciousness for those things either (I’m thinking primarily of quantum mechanics).

    ‘Course, this doesn’t make us automatons. Our “spirit” self is simply the “random number generator”, so to speak.

    Like this comment? Thumb up 0

  3. Bruce on January 13, 2011 at 8:19 PM

    “Why couldn’t one model consciousness with an intricately nested hierarchy of values/objectives/rules and a random number generator?”

    That is definitely a possibility, isn’t it?

    Like this comment? Thumb up 0

  4. Mark D. on January 14, 2011 at 10:29 PM

    If consciousness can be explained, then it can be reduced to an algorithm. Doesn’t that mean we’re all just automatons? But if we say consciousness is not algorithmic, then doesn’t that mean we are claiming consciousness is fundamentally unexplainable?

    I think you are misusing the term “explained” here. Granted though, this is the root of the debate about libertarian free will, which in my opinion is the touchstone for nearly all serious philosophical questions.

    Like this comment? Thumb up 0

  5. Bruce on January 15, 2011 at 9:51 AM

    Mark D,

    Consider my point of view (at the moment at least) to be that ‘explained’ always implies ‘we can make an algorithm out of it.’

    I do not believe an explanation that does not result in algorithms can truly be said to be ‘explained’ fully. I can’t think of counter examples to this unless we’re talking about purely subjective things. Then the explanation is always ‘it’s a personal taste.’

    This is the point of view I have been defending in my posts. Though for all I know, next year I’ll have a new point of view.

    It is interesting that you mention that all serious philosphical questions arise from the question of libertarian free will. I guess I hadn’t really consider that since I’m not interested in philosophy very much, so I’m not sure what the serious questions are.

    Like this comment? Thumb up 0

  6. Mark D. on January 16, 2011 at 8:46 AM

    Bruce, arguendo, let me introduce an issue from classical theism. The debate is about which comes first, God’s will or his reason. If his will, then his will is unreasoned. If his reason, then his reason is unwilled.

    So you have the suggestion where either God’s nature is arbitrary or you have the suggestion that God’s will is determined by his nature. In one case the laws of nature and natural law (of morality) are arbitrary and unreasoned, in the other case God’s will is dictated by them. The endless regress should be obvious.

    Similarly if a great composer sits down and authors a piano concerto, to whom or what are we to credit its beauty and sophistication? Because if there is no libertarian free will, we certainly cannot credit the composer’s creativity.

    So do we credit the laws of nature then? Is every great music composition just a reflection of Schroedinger’s equation? Or is it pure happenstance? Or a combination of the two?

    The great philosophical problem with stochasto-deterministic evolution as the origin of all things revolves around the same question. So far as we can tell, the fundamental laws of nature can be written down on a napkin.

    So you start with a random ball of gas, and what is written down on a napkin, and you can thereby explain the inevitable advent of something as sophisticated as human culture and civilization? One where the only difference between a Mozart concerto and a Beethoven symphony is random chance?

    That is basically the fundamentalist evolutionary point of view – everything is traceable to an accident. Where meaning, intention, purpose, creativity, morality, free will, identity etc are epiphenomena at best.

    Like this comment? Thumb up 0

  7. Bruce on January 16, 2011 at 2:05 PM

    I can see why I’m not interested in philosphy very much. Trying to determine ‘which came first, God’s will or His reason’ is about as pointless as things can get. ;)

    And I think deciding Mozart and Beethoven are just random chance is to misunderstand the word ‘random.’

    The problem with calling “meaning, intention, purpose, creativity, morality, free will, identity” all “epiphenomena” is that this assumes that physical reducibility is somehow fundamental. It is not.

    In any case, if these are the serious questions of philosphy (and I believe you are right), then I guess my feeling is that philosphy shouldn’t be taken seriously. :P (Or at least not these questions.)

    These Philosphy questions seems to be asking the wrong questions based on faulty assumptions. For example, it seems patently obvious to me that any time you are speaking of a conscious being (including God) that reason and will are both part of what it means to be conscious. Therefore, the question is invalid at the outset.

    Like this comment? Thumb up 0

  8. Mark D. on January 17, 2011 at 8:50 AM

    Physical reducibility is, strictly speaking, a mathematical concept. Reductionism does not work in reverse. If you start with a bunch of particles in Hamiltonian orbits, there is no way to apply a mathematical transformation and reliably produce an explanation in terms of a non-physical theory.

    We say then that all biological entities are physical, but all physical entities are not biological (so far as we can tell). You cannot explain electron orbits in terms of molecular genetics, but you can do the reverse.

    The former is more fundamental than the latter because the transformation in one direction is information preserving and the other one is not. Any information preserving transformation of a physical model is going to be just another physical model.

    Like this comment? Thumb up 0

  9. Mark D. on January 17, 2011 at 9:10 AM

    As far as philosophy goes, I pulled out an admittedly obscure debate from medieval philosophy to make a point about free will and meaning, not to suggest the issue as framed is relevant within the scheme of LDS theology.

    As far as the degree to which Mozart’s compositions are necessarily “random” from a ordinary physicalist point of view, I suggest you need to be more aggressive about applying mathematical transformations.

    Causality and determinism are best expressed as problems in information theory. So the question is where did the information in Mozart’s compositions come from? On the random ball of gas model of the world there are only three possibilities – from the laws of physics, from random initial conditions, or from random events in between, or any combination thereof.

    All non-random events (as in classical mechanics) are information preserving, i.e. they are not information sources at all. In fact from a classical point of view Mozart’s compositions have to be written in the stars, because there has never been an information producing event ever. If you cannot see Mozart’s composition in the structure of the priomordial ball of gas, it is because your telescope isn’t advanced enough.

    Of course if QM injects actual randomness (more entropy than information) into the system on a regular basis, then random wave collapses are certainly a possible alternative as well. But what you cannot do, in a stochasto-deterministic view of the universe, is ever trace Mozart’s compositions uniquely to anything special about Mozart. That is what it means for Mozart to be epiphenomenal, as in unnecessary to a complete explanation of everything that he ever wrote.

    Like this comment? Thumb up 0

  10. Mark D. on January 17, 2011 at 9:16 AM

    “All non-random events (as in classical mechanics) are information preserving”

    Correction: that should be “all causally determined events”. LFW events are non-random information sources, by definition. Of course many dispute whether there is such a thing. LFW is like the second law of thermodynamics in reverse.

    Like this comment? Thumb up 0

  11. Bruce on January 17, 2011 at 7:11 PM

    “That is what it means for Mozart to be epiphenomenal, as in unnecessary to a complete explanation of everything that he ever wrote.”

    I don’t dispute what you are saying. Most likely the ball of gas, as you are calling it, can explain everything in some sense. (Exploring all possiblities through MWI maybe?)

    Nor am I disputing that Mozart can’t legitimately be called “epiphenomenal.”

    What I doubt is that that is a useful way to looking at Mozart. True, but useless.

    Instead, it’s more useful to think of him as a conscious being reacting to his environment. Then we can ask what a conscious being is. Then we can look at how the ‘software’ for consciousness works. Then we can break that down to the laws of computation. Then we understand it! — And all without reducing to physics.

    The problem is that “epiphenomenal” seems like a negative connotation to me. Sure biology is “epiphenomenal” of physics. But it is also a fact that we can’t trace biology to physics all that well and probably never will. Nor will we have much need to, because physical reducibility jus isn’t that important. After all, we can still create a science of biology without being able to reduce to physics, right?

    This is because biology is — like everything — reducible to computation.

    That reduction to computation is way more useful than the hypothetical reduction to physics. In fact, this is usually (and almost always) true. Therefore, there is a sense in which the laws of computation are actually more fundamental to explanation than the laws of physics. All explanations reduce to computation, so far as we known. As in 100%. If we can’t reduce it to computation, we don’t really understand it.

    All things reduce to physics too, but we generally don’t care except in principle.

    Like this comment? Thumb up 0

  12. Mark D. on January 18, 2011 at 9:54 AM

    I agree that all things are representable in physical or natural terms, but I wildly disagree with most conventional physicists about what those natural terms are. LFW isn’t based on anything physics as we know it today contemplates.

    Physics as we know it today can’t explain perception, purpose, intention, or any of those other things mostly philosophers worry about.

    Like this comment? Thumb up 0

  13. Bruce on January 18, 2011 at 6:39 PM

    “Physics as we know it today can’t explain perception, purpose, intention, or any of those other things mostly philosophers worry about.”

    I’m curious. Have you read Godel, Escher, Bach: An Eternal Golden Braid?

    I was planning to eventually do some posts on his ideas. That is a scientific approach that addresses those very items, but through computation rather than physics.

    I am not sure if I should buy into such ideas or not.

    Like this comment? Thumb up 0

  14. Our Epistemology So Far | Wheat and Tares on March 24, 2011 at 3:31 PM

    [...] the primary means of gaining new knowledge is through coming up with conjectures (theories) and then over time refuting them. This forms a ‘natural selection’ or ‘survival of the fittest’ of ideas and explanations of [...]

    Like this comment? Thumb up 0

  15. [...] of explanations, namely ones that have survived the strongest criticisms and are highly (preferably computationally) specific and hard to vary, yet is always open to new ideas or insight no matter what the [...]

    Like this comment? Thumb up 1

  16. [...] of explanations, namely ones that have survived the strongest criticisms and are highly (preferably computationally) specific and hard to vary, yet is always open to new ideas no matter what the [...]

    Like this comment? Thumb up 0

  17. [...] of explanations, namely ones that have survived the strongest criticisms and are highly (preferably computationally) specific and hard to vary, yet is always open to new ideas no matter what the [...]

    Like this comment? Thumb up 0

Leave a Reply

Subscribe without commenting

Archives

%d bloggers like this: