FOA Home | UP: Looking for meaning in all the wrong places \\ (at the character level)

Benoit Mandelbrot's explanation

The early days of cybernetics were heady, and Zipf was not alone in seeking a grand, unifying theory that might explain the phenomena of communication on computational grounds like those proving so successful in physics. Benoit Mandelbrot was equally ambitious.

Mandelbrot's background as a physicist is clear when he considers the message decoder as a physical piece of apparatus, ``... cutting a continuous incoming string of signs into groups, and recoding each group separately.'' This ``differentiator'' complements an ``integrator'' which reconstitutes new messages from individual words. Within this model communication can be considered ``fully analogous to the perfect gas of thermodynamics.'' Minimizing the cost of transmission corresponds to minimization of free energy in thermodynamics.

Mandelbrot was also interested in how the critical parameter \alpha \) varied from one vocabulary to another. Extending the physical analogy of thermodynamic energy, the ``informational temperature'' or ``temperature of discourse'' is proportional to \( 1/\alpha \), which Mandelbrot argues provides a much better measure of the richness of a vocabulary than simply counting the number of words it contains.

The value 1/\alpha \) can also be used to relate our analysis of Zipf's Law to Mandelbrot's fractals. If the letters of our alphabet are imagined to be digits of numbers base n + 1, and a leading decimal point is placed before each word. Then each word corresponds to a number between 0 and 1. \bq The construction amounts in effect to cutting out of [0, 1] all the numbers that include the digit 0 otherwise than at the end. One finds that the remainder is a Cantor dust, the fractal dimension of which is precisely \( 1/\alpha \) . [] (p. 346) [] (p. 346) ] (p. 346) (p. 346) . 346) 346) 46) ) \eq

Mandelbrot proposed a more general form of Zipf's Law: F(r)={C\over{(r+b)^\alpha}} that has proved important to analysis of the relationship between word frequencies and their rank (cf. Section § ).

Mandelbrot also suggested how this model might be applied within a model of cognition: Whatever the detailed structure of the brain it recodes information many times. The public representation through phonemes is immediately replaced by a private one through a string of nerve impulses.... This recorded message presumably uses fewer signs than the incoming one; therefore when a given message reaches a higher level it will have been reduced to a choice between a few possibilities only without the extreme redundancy of the sounds. The last stages are ``idea'' stages, where not only the public representation has been lost, but also the public elements of information. (p. 488- 489) \eq He also makes other provocative suggestions, for example that schizophrenics provide the best test of his theory since these individuals impose fewest ``semantic constraints'' on the random process (generating language) of interest?!

While unsuccessful at his more ambitious goals of wedding a physical model of communcation to models of semantics such as Saussure's, Mandelbrot was probably the first to characterize the real truth underlying Zipfian distributions. Before considering this derivation, one more historical perspective, due to Herbert Simon, will be mentioned.

Top of Page | UP: Looking for meaning in all the wrong places \\ (at the character level) | ,FOA Home

FOA © R. K. Belew - 00-09-21