Longest common substring

In computer science, a longest common substring of two or more strings is a longest string that is a substring of all of them. There may be more than one longest common substring. Applications include data deduplication and plagiarism detection.

## Contents

* 1 Examples
* 2 Problem definition
* 3 Algorithms
* 3.1 Suffix tree
* 3.2 Dynamic programming
* 4 See also
* 5 References
* 6 External links

## Examples

The strings "BADANAT" and "CANADAS" share the maximal-length substrings "ADA" and "ANA".

The picture shows two strings where the problem has multiple solutions. Although the substring occurrences always overlap, no longer common substring can be obtained by "uniting" them.

The strings "ABABC", "BABCA" and "ABCBA" have only one longest common substring, viz. "ABC" of length 3. Other common substrings are "A", "AB", "B", "BA", "BC" and "C".

ABABC
|||
BABCA
|||
ABCBA

## Problem definition

Given two strings, [math]\displaystyle{ S }[/math] of length [math]\displaystyle{ m }[/math] and [math]\displaystyle{ T }[/math] of length [math]\displaystyle{ n }[/math], find a longest string which is substring of both [math]\displaystyle{ S }[/math] and [math]\displaystyle{ T }[/math].

A generalization is the k-common substring problem. Given the set of strings [math]\displaystyle{ S = \\{S_1, ..., S_K\\} }[/math], where [math]\displaystyle{ |S_i|=n_i }[/math] and [math]\displaystyle{ \Sigma n_i = N }[/math]. Find for each [math]\displaystyle{ 2 \leq k \leq K }[/math], a longest string which occurs as substring of at least [math]\displaystyle{ k }[/math] strings.

## Algorithms

One can find the lengths and starting positions of the longest common substrings of [math]\displaystyle{ S }[/math] and [math]\displaystyle{ T }[/math] in [math]\displaystyle{ \Theta }[/math][math]\displaystyle{ (n+m) }[/math] time with the help of a generalized suffix tree. A faster algorithm can be achieved in the word RAM model of computation if the size [math]\displaystyle{ \sigma }[/math] of the input alphabet is in [math]\displaystyle{ 2^{o(\sqrt{\log(n+m)})} }[/math]. In particular, this algorithm runs in [math]\displaystyle{ O((n+m)\log\sigma/\sqrt{\log (n+m)}) }[/math] time using [math]\displaystyle{ O((n+m)\log\sigma/\log (n+m)) }[/math] space.[1] Solving the problem by dynamic programming costs [math]\displaystyle{ \Theta(nm) }[/math]. The solutions to the generalized problem take [math]\displaystyle{ \Theta(n_1 + ... + n_K) }[/math] space and [math]\displaystyle{ \Theta(n_1 }[/math]·...·[math]\displaystyle{ n_K) }[/math] time with dynamic programming and take [math]\displaystyle{ \Theta((n_1 + ... + n_K) * K) }[/math] time with a generalized suffix tree.

### Suffix tree

Generalized suffix tree for the strings "ABAB", "BABA" and "ABBA", numbered 0, 1 and 2.

The longest common substrings of a set of strings can be found by building a generalized suffix tree for the strings, and then finding the deepest internal nodes which have leaf nodes from all the strings in the subtree below it. The figure on the right is the suffix tree for the strings "ABAB", "BABA" and "ABBA", padded with unique string terminators, to become "ABAB$0", "BABA$1" and "ABBA$2". The nodes representing "A", "B", "AB" and "BA" all have descendant leaves from all of the strings, numbered 0, 1 and 2.

Building the suffix tree takes [math]\displaystyle{ \Theta(N) }[/math] time (if the size of the alphabet is constant). If the tree is traversed from the bottom up with a bit vector telling which strings are seen below each node, the k-common substring problem can be solved in [math]\displaystyle{ \Theta(NK) }[/math] time. If the suffix tree is prepared for constant time lowest common ancestor retrieval, it can be solved in [math]\displaystyle{ \Theta(N) }[/math] time.[2]

### Dynamic programming

The following pseudocode finds the set of longest common substrings between two strings with dynamic programming:

function LongestCommonSubstring(S[1..r], T[1..n])
L := array(1..r, 1..n)
z := 0
ret := {}

for i := 1..r
for j := 1..n
if S[i] = T[j]
if i = 1 or j = 1
L[i, j] := 1
else
L[i, j] := L[i − 1, j − 1] + 1
if L[i, j] > z
z := L[i, j]
ret := {S[i − z + 1..i]}
else if L[i, j] = z
ret := ret ∪ {S[i − z + 1..i]}
else
L[i, j] := 0
return ret

This algorithm runs in [math]\displaystyle{ O(n r) }[/math] time. The array `L` stores the length of the longest common substring of the prefixes `S[1..i]` and `T[1..j]` which end at position `i` and `j`, respectively. The variable `z` is used to hold the length of the longest common substring found so far. The set `ret` is used to hold the set of strings which are of length `z`. The set `ret` can be saved efficiently by just storing the index `i`, which is the last character of the longest common substring (of size z) instead of `S[i-z+1..i]`. Thus all the longest common substrings would be, for each i in `ret`, `S[(ret[i]-z)..(ret[i])]`.

The following tricks can be used to reduce the memory usage of an implementation:

* Keep only the last and current row of the DP table to save memory ([math]\displaystyle{ O(\min(r, n)) }[/math] instead of [math]\displaystyle{ O(n r) }[/math])
* The last and current row can be stored on the same 1D array by traversing the inner loop backwards
* Store only non-zero values in the rows. This can be done using hash-tables instead of arrays. This is useful for large alphabets.

## See also

* Longest palindromic substring
* n-gram, all the possible substrings of length n that are contained in a string

## References

1. ↑ Charalampopoulos, Panagiotis; Kociumaka, Tomasz; Pissis, Solon P.; Radoszewski, Jakub (Aug 2021). "Faster Algorithms for Longest Common Substring". in Mutzel, Petra; Pagh, Rasmus; Herman, Grzegorz. European Symposium on Algorithms. 204. Schloss Dagstuhl. doi:10.4230/LIPIcs.ESA.2021.30. Here: Theorem 1, p.30:2.
2. ↑ Gusfield, Dan (1999). Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology. USA: Cambridge University Press. ISBN 0-521-58519-8. https://books.google.com/books?id=Ofw5w1yuD8kC&q="longest+common+substring".

## External links

* Dictionary of Algorithms and Data Structures: longest common substring
* Perl/XS implementation of the dynamic programming algorithm
* Perl/XS implementation of the suffix tree algorithm
* Dynamic programming implementations in various languages on wikibooks
* working AS3 implementation of the dynamic programming algorithm
* Suffix Tree based C implementation of Longest common substring for two strings

* v
* t
* e

Strings

String metric|

* Approximate string matching
* Bitap algorithm
* Damerau–Levenshtein distance
* Edit distance
* Gestalt Pattern Matching
* Hamming distance
* Jaro–Winkler distance
* Lee distance
* Levenshtein automaton
* Levenshtein distance
* Wagner–Fischer algorithm

String-searching algorithm|

* Apostolico–Giancarlo algorithm
* Boyer–Moore string-search algorithm
* Boyer–Moore–Horspool algorithm
* Knuth–Morris–Pratt algorithm
* Rabin–Karp algorithm

Multiple string searching|

* Aho–Corasick
* Commentz-Walter algorithm

Regular expression|

* Comparison of regular expression engines
* Regular tree grammar
* Thompson's construction
* Nondeterministic finite automaton

Sequence alignment|

* Hirschberg's algorithm
* Needleman–Wunsch algorithm
* Smith–Waterman algorithm

Data structures|

* DAFSA
* Suffix array
* Suffix automaton
* Suffix tree
* Generalized suffix tree
* Rope
* Ternary search tree
* Trie

Other|

* Parsing
* Pattern matching
* Compressed pattern matching
* Longest common subsequence
* Longest common substring
* Sequential pattern mining
* Sorting

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Longest common substring. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Longest_common_substring&oldid=3014986"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template