LZD Factorization: Simple and Practical Online Grammar Compression with Variable-to-Fixed Encoding

Abstract

We propose a new variant of the LZ78 factorization which we call the LZ Double-factor factorization (LZD factorization). Each factor of the LZD factorization of a string is the concatenation of the two longest previous factors, while each factor of the LZ78 factorization is that of the longest previous factor and the following character. Interestingly, this simple modification drastically improves the compression ratio in practice. We propose two online algorithms to compute the LZD factorization in $O(m (M + \min (m, M)\log \sigma ))$ time and $O(m)$ space, or in $O(N \log \sigma )$ time and $O(N)$ space, where $m$ is the number of factors to output, $M$ is the length of the longest factor(s), $N$ is the length of the input string, and $\sigma$ is the alphabet size. We also show two versions of our LZD factorization with variable-to-fixed encoding, and present online algorithms to compute these versions in $O(N + \min (m, 2^L) (M + \min (m, M, 2^L) \log \sigma ))$ time and $O(\min (2^L, m))$ space, where $L$ is the bit-length of each fixed-length code word. The LZD factorization and its versions with variable-to-fixed encoding are actually grammar-based compression, and our experiments show that our algorithms outperform the state-of-the-art online grammar-based compression algorithms on several data sets.