First Pass HoloDynamic Compression of Chaotic Data
To begin doing data ring analysis on chaotic data one needs to construct a sample of chaotic data. By chaotic data I mean data that has some structure but appears to be random, that is data that has very high informational entropy. A good example might be what you have left over when your drop a vase and it shatters on the floor. There is still structure and pattern in what you have left over, but it also appears to be a random collection of the pieces of what used to be a vase. The fact is one could rebuild a reasonable fascimile of the orignal vase from the pieces. A good analogy to this in terms of data is the data that results after a file has been compressed by some conventional data compression technique. What is left still must have some structure because one needs to be able to construct some fascimile of the orignal file from it, but it appears to be random.The files I had chosen for this test case were the results of files compressed twice by highly efficient conventional data compression techniques. The original files were digitized movies of a few minutes in length each . These files were then compressed by the MPEG I compression technique, a form of video compression. The files were then further compressed using a technique known as higher order arithmetic coding. Arithmetic coding is a very efficient way of removing redundancies from data. What was left were files whose informational entropy was very close to that of files produced by random number generators.
To begin to examine data rings in the data stream of such a file first one needs to decide on the size of the data rings to look at. Do you look at data rings of 1000 bytes, 100 bytes or 10 bytes? Secondly one needs to actually shift the rings(see "Data Rings in HoloDynamic Compression") to each position relative to itself and look for matching data.
I undertook a systematic search of numberous samples of data rings ranging in size from 200000 bytes to 16 bytes. The best results came about for data rings of 64 bytes in length. To allow for statistical fluctuations I looked at approximately fifty thousand data rings of 64 bytes in length from two different files of chaotic(arithmetically encoded MPEG in this case) data. In both files it was found that there was sufficient data ring matching for further data compression to result.
What do I mean by "sufficient" data ring matching? First, one needs to be aware that matching in a data ring exists because of redundancies in the ring. For example, if you shift the eight byte ring "CXVCGHIM" three bytes to become "HIMCXVCG" and compare the original ring (CXVCGHIM) with the shifted ring (HIMCXVCG) one can see that there is a matching "C" in the fourth place. Let's say we remove the "C" from the fourth place in the original string. Now we have a ring of data that is only seven bytes long. By using english we can say "shift the ring three bytes and insert the byte that is in the fourth place back into the ring". This will return the ring to its original form. How would one express this in digital form? Clearly there would need to be two parameters. One, the number of bytes the ring is shifted and two, which byte to insert back into the original ring (the matching byte that was removed). The number of bytes that the ring can be shifted can be a number anywhere between 1 and 7. Zero and eight simply leave the ring in its original form. The number of places to reinsert the matching byte go from position zero(the first byte in the ring) to position 7(the last byte in the ring). In both cases the parameter can be expressed by three bits. This will allow one to express 2 times 2 times 2 consecutive numbers from 0 to seven. So we can express the sentence "shift the ring three bytes and insert the byte that it is in the fourth place back into the ring" in the form of two three bit parameters or six bits total, in this case 011(three) and 100(four). By removing the matching byte from the ring, we have removed one byte or 8 bits from the ring(forgetting about the fact that we are using ASCII letters in our example). We can describe how to return the ring to its original form using only 6 bits so we have a net compression of 8-6 = 2 bits.
The procedure that I used to determine "sufficient" data ring matching was more complicated than the procedure explained above but the basic concept was the same. Using this procedure I was able to produce further compression in these chaotic files. What happens if I repeat the same process on a file that has already been compressed using this procedure? This will be the subject for the next article on HoloDynamic Compression. It should be noted that I applied this same procedure to a file that had been generated using an efficient random number generator. This file had higher informational entropy or less redundancies than the chaotic files that I used in this test case. The first pass compression result was very similar to that obtained from the chaotic files. This suggests that another concept needs to be used to talk about randomness or lack of patterns in data beyond the concept of informational entropy.
The ideas and concepts linking mind and metaphysics in HoloDynamics and HoloDynamic Compression have resulted in software with practical applications. See Holoentropy and Multipass HoloDynamic Compression: Steps to a Prototype.
My research into HoloDynamic Compression has led to the general study of HoloDynamics. See Fundamental Concepts of HoloDynamics.
Questions, comments, collaborations, all are welcome with Aloha!
More information on HoloDynamic Compression will be available shortly. Presentations on HoloDynamic Compression are available if you are visiting Maui or if arrangements can be made for a presentation at your site.
Click here to return to "The Celestine Prophecy" Home Page.