Sequitur. CSEP 590 Data Compression Autumn Context-Free Grammars. Context-Free Grammar Example. Arithmetic Expressions

Transcription

1 equitur CEP 590 Data Compression utumn 2007 equitur Nevill-Manning and Witten, Uses a context-free grammar (without recursion) to represent a string. The grammar is inferred from the string. If there is structure and repetition in the string then the grammar may e very small compared to the original string. Clever encoding of the grammar yields impressive compression ratios. Compression plus structure! CEP Lecture 5 - utumn Context-Free Grammars Invented y Chomsky in 1959 to explain the grammar of natural languages. lso invented y Backus in 1959 to generate and parse Fortran. Example: terminals:, e non-terminals:, Production Rules:,, e, is the start symol Context-Free Grammar Example e Example: and e matched as parentheses derivation of eee e e e ee eee hierarchical parse tree e e e CEP Lecture 5 - utumn CEP Lecture 5 - utumn T T T T*F T F F a F () rithmetic Expressions derivation of a * (a + a) + a +T T+T T*F+T F*F+T a*f+t a*()+f a*(+f)+t a*(t+f)+t a*(f+f)+t a*(a+f)+t a*(a+a)+t a*(a+a)+f a*(a+a)+a a T T * F + T F ( ) a + T T F parse tree F a F a tring x Overview of Grammar Compression Encoder Grammar inference Context-free grammar Grammar encoding ymol stream Entropy coder Compressed it stream Decoder Grammar derivation Context-free grammar Grammar decoding ymol stream Entropy decoder x CEP Lecture 5 - utumn CEP Lecture 5 - utumn

2 equitur Principles Digram Uniqueness: no pair of adjacent symols (digram) appears more than once in the grammar. Rule Utility: Every production rule is used more than once. These two principles are maintained as an invariant while inferring rammar for the input string. equitur Example (1) CEP Lecture 5 - utumn CEP Lecture 5 - utumn equitur Example (2) equitur Example (3) e CEP Lecture 5 - utumn CEP Lecture 5 - utumn equitur Example (4) equitur Example (5) e ee e occurs twice. Create new rule. CEP Lecture 5 - utumn CEP Lecture 5 - utumn

3 equitur Example (6) equitur Example (7) e CEP Lecture 5 - utumn CEP Lecture 5 - utumn equitur Example (8) equitur Example (9) e ee e occurs twice. Use existing rule. CEP Lecture 5 - utumn CEP Lecture 5 - utumn equitur Example (10) equitur Example (11) e e CEP Lecture 5 - utumn CEP Lecture 5 - utumn

4 equitur Example (12) equitur Example (13) ee e occurs twice. Use existing rule. e Enforce digram uniqueness occurs twice. Create new rule. CEP Lecture 5 - utumn CEP Lecture 5 - utumn equitur Example (14) equitur Example (15) BeB BeB CEP Lecture 5 - utumn CEP Lecture 5 - utumn equitur Example (16) equitur Example (17) BeB BeBe e occurs twice. Use existing rule. CEP Lecture 5 - utumn CEP Lecture 5 - utumn

5 equitur Example (18) BeB equitur Example (19) -> BeB -> e B -> CEP Lecture 5 - utumn CEP Lecture 5 - utumn equitur Example (20) equitur Example (21) BeBe e occurs twice. Use existing rule. BeB occurs twice. Use existing rule. CEP Lecture 5 - utumn CEP Lecture 5 - utumn equitur Example (22) equitur Example (23) BeBB B occurs twice. Create new rule C B. CeBC C B CEP Lecture 5 - utumn CEP Lecture 5 - utumn

6 equitur Example (24) equitur Example (25) CeBCe C B Ce occurs twice. Create new rule D Ce. DBD C B D Ce Enforce rule utility. C occurs only once. Remove C B. CEP Lecture 5 - utumn CEP Lecture 5 - utumn equitur Example (26) DBD D Be The Hierarchy DBD D Be D B D B e B e e e e e e e Is there compression? In this small example, proaly not. CEP Lecture 5 - utumn CEP Lecture 5 - utumn equitur lgorithm Input the first symol s to create the production s; repeat match an existing rule:.xy.b. B XY B XY create a new rule:.xy..c... B...XY... B...C... remove a rule: C XY.B. B X 1 X 2 X. X 1 X 2 X k. k input a new symol: X 1 X 2 X k until no symols left X 1 X 2 X k s CEP Lecture 5 - utumn Exercise Use equitur to construct rammar for aaaaaaaaaa = a 10 CEP Lecture 5 - utumn

7 Complexity The numer of non-input sequitur operations applied < 2n where n is the input length. ince each operation takes constant time, sequitur is a linear time algorithm mortized Complexity rgument Let m = # of non-input sequitur operations. Let n = input length. how m 2n. Let s = the sum of the right hand sides of all the production rules. Let r = the numer of rules. We evaluate 2s - r. Initially 2s - r = 1 ecause s = 1 and r = 1. 2s - r > 0 at all times ecause each rule has at least 1 symol on the right hand side. CEP Lecture 5 - utumn CEP Lecture 5 - utumn equitur Rule Complexity Digram Uniqueness - match an existing rule..xy B XY.B. B XY Digram Uniqueness - create a new rule..xy. B...XY....C... B...C... C XY Rule Utility - Remove a rule. s r -1 0 s r 0 1 2s - r -2 2s - r -1 mortized Complexity rgument 2s - r > 0 at all times ecause each rule has at least 1 symol on the right hand side. 2s - r increases y 2 for every input operation. 2s - r decreases y at least 1 for each non-input sequitur rule applied. n = numer of input symols m = numer of non-input operations 2n m > 0. m 2n..B. B X 1 X 2 X k s r. X 1 X 2 X k s - r -1 CEP Lecture 5 - utumn CEP Lecture 5 - utumn mortized Complexity rgument Input Non-input s - r 10 5 Linear Time lgorithm There is a data structure to implement all the sequitur operations in constant time. Production rules in an array of douly linked lists. Each production rule has reference count of the numer of times used. Each nonterminal points to its production rule. Digrams stored in a hash tale for quick lookup. 0 Time CEP Lecture 5 - utumn CEP Lecture 5 - utumn

8 CeBCe C B Data tructure Example 0 C e B C e 2 B 2 e C 2 B reference count current digram digram tale BC eb Ce e B CEP Lecture 5 - utumn Grammar Basic Encoding a Grammar DBD D Be ymol Code Grammar Code 000 e B 011 D 100 # 101 D B D # e # # B e its Grammar Code = ( s + r 1) log2( r + a) r = numer of rules s = sum of right hand sides a = numer in original symol alphaet No code for needed CEP Lecture 5 - utumn Better Encoding of the Grammar Nevill-Manning and Witten suggest a more efficient encoding of the grammar that uses LZ77 ideas. end the right hand side of the production. The first time a nonterminal is sent, its right hand side is transmitted instead. The second time a nonterminal is sent as a triple [i, j, d], which says the right hand side starts at position j in production rule i and is d long. new production rule is then added to a dictionary. usequently, the nonterminal is represented y the index of the production rule. CEP Lecture 5 - utumn tbca BBB t B C a T tagt X 0 tagt l 0 = 4 CEP Lecture 5 - utumn tbca BBB T tagt [0, 1, 3] X 0 t X 1 X 1 l 0 = 3 X 1 agt l 1 = 3 tbca BBB T tagt [0, 1, 3] 1 X 0 t X 1 X 1 X 1 l 0 = 4 X 1 agt l 1 = 3 t B C a t B C a CEP Lecture 5 - utumn CEP Lecture 5 - utumn

9 tbca BBB T tagt [0, 1, 3] 1 [0, 1, 3] X 0 t X 2 X 2 l 0 = 3 X 1 agt l 1 = 3 X 2 X 1 X 1 X 1 l 2 = 3 tbca BBB T tagt [0, 1, 3] 1 [0, 1, 3] 1 X 0 t X 2 X 2 X 1 l 0 = 4 X 1 agt l 1 = 3 X 2 X 1 X 1 X 1 l 2 = 3 t B C a t B C a CEP Lecture 5 - utumn CEP Lecture 5 - utumn tbca BBB t B C a T tagt [0, 1, 3] 1 [0, 1, 3] 1 [1, 0, 2] X 0 t X 2 X 2 X 1 X 3 l 0 = 5 X 1 X 3 t l 1 = 2 X 2 X 1 X 1 X 1 l 2 = 3 X 3 ag l 3 = 2 tbca BBB t B C a T tagt [0, 1, 3] 1 [0, 1, 3] 1 [1, 0, 2] a X 0 t X 2 X 2 X 1 X 3 a l 0 = 6 X 1 X 3 t l 1 = 2 X 2 X 1 X 1 X 1 l 2 = 3 X 3 ag l 3 = 2 CEP Lecture 5 - utumn CEP Lecture 5 - utumn Kieffer-Yang Improvement Kieffer and Yang developed a theoretical framework for studying these types of grammars in KY is universal; it achieves entropy in the limit dd to sequitur Reduction Rule 5: B CD B ae C a D cd E D <> = <B> = acd CD B ae C a D cd E D dding this constraint makes sequitur universal. CEP Lecture 5 - utumn Compression Quality Neville-Manning and Witten 1997 size comp gzip sequitur PPMC i ook geo oj pic progc = First; = econd; = Third. Files from the Calgary Corpus Units in its per character (8 its) compress - ased on LZW gzip - ased on LZ77 PPMC - adaptive arithmetic coding with context zip2 Burrows-Wheeler lock sorting zip CEP Lecture 5 - utumn

10 Notes on equitur Yields compression and hierarchical structure simultaneously. With clever encoding is competitive with the est of the standards. The grammar size is not close to approximation algorithms Upper = O((n/log n) 3/4 ); Lower = Ω(n 1/3 ). (Lehman, 2002) But! Practical linear time encoding and decoding. Other Grammar Based Methods Longest Match Most frequent digram Match producing the est compression CEP Lecture 5 - utumn CEP Lecture 5 - utumn