Index of Coincidence for the Vigenère Cipher
Pick two letters at random from a long English passage. The chance that both letters are the same is noticeably higher than it would be if A–Z occurred with equal probability.
That difference exists because natural language is uneven. Letters such as E, T, A, and O occur far more often than letters such as Q and Z.
The Index of Coincidence, usually abbreviated IC or IoC, measures this unevenness as the probability that two letters selected from a text are identical.
For Vigenère cryptanalysis, that simple statistic becomes useful because a correct guess about the repeating-key period separates the ciphertext into streams whose letter distributions regain the unevenness expected from ordinary language.
The Index of Coincidence does not reveal a Vigenère key by itself. Its real value is diagnostic: it helps an analyst decide whether a proposed period is consistent with the statistical structure expected from the plaintext.
If you already know the key and simply want to decrypt a message, use the Vigenère Cipher Decoder and Encoder. IC becomes relevant when the key or its period is unknown.
What the Index of Coincidence actually measures
Imagine a text containing N letters.
Select one letter at random, then select a second letter from the remaining characters. The Index of Coincidence asks:
It is not counting how often identical letters appear next to each other.
A text containing no double letters such as EE can still have a high IC. The two matching letters can come from completely different positions in the text.
What matters is the overall frequency distribution.
If one or two letters occur very often, there are many possible pairs consisting of those same letters. If all 26 letters occur almost equally often, matching pairs are less likely.
The Index of Coincidence formula
Let fA, fB, through fZ represent the number of times each letter appears, and let N be the total number of letters.
The raw Index of Coincidence is:
The calculation has a direct probability interpretation.
Suppose the letter E appears fE times. The probability that the first selected character is E is:
After one E has been selected, there are fE – 1 E characters left among N – 1 remaining characters:
Multiplying them gives the probability of drawing E twice. Repeat that calculation for every letter and add the probabilities.
The same formula can also be written in terms of pairs:
Both expressions calculate the same quantity.
Why English and random letters have different IC values
If the 26 letters of the English alphabet occurred with exactly equal probability, each letter would have probability 1/26.
The expected coincidence probability would therefore be approximately:
Natural English is not uniform. Its uneven frequencies raise the probability that two independently selected letters will match.
English text is often described as having a raw IC around 0.065 to 0.068, although the exact value changes with the corpus, writing style, vocabulary, and sample length.
| Letter distribution | Typical raw IC | Interpretation |
|---|---|---|
| Uniform A–Z distribution | ≈ 0.0385 | Every letter is equally likely. |
| Ordinary English | Often ≈ 0.065–0.068 | Unequal language frequencies create more matching pairs. |
| Real short sample | Can vary widely | Sampling noise can dominate the expected language value. |
These values should be treated as reference points rather than rigid thresholds.
A technical passage filled with unusual terminology can have a different IC from a novel. A 30-letter sample can differ dramatically from the long-term English expectation simply because it contains too little text.
Why a Caesar shift does not change the IC
This property explains why IC is so useful against Vigenère.
Suppose an English plaintext contains:
Apply a Caesar shift of 7. The identities of the letters change:
But their counts do not.
A Caesar shift merely relabels the frequency distribution.
The IC formula only cares how many times each symbol occurs, not which letter owns a particular count. Rearranging those counts among the alphabet therefore leaves the Index of Coincidence unchanged.
This is the critical connection to Vigenère: once characters encrypted with the same Vigenère key position are isolated, that stream is essentially a Caesar-shifted sample of the plaintext.
Why Vigenère lowers the IC of the combined ciphertext
A repeating-key Vigenère cipher does not use one Caesar shift. It cycles through several.
Suppose the key is:
Its shifts are:
A frequent plaintext letter such as E is no longer moved to one ciphertext letter. Depending on the key position, it can be shifted to several different symbols.
The uneven English frequency distribution is therefore spread across multiple shifted alphabets.
When all of those alphabets are mixed together, the overall ciphertext tends to look flatter than ordinary English. Its IC often moves closer to the uniform-alphabet value.
It normally does not become perfectly uniform, and the exact whole-text IC depends on the key period, sample length, plaintext distribution, and key values.
The useful observation is not that “Vigenère always has IC 0.04.” It is that mixing several shifted language distributions generally reduces the coincidence signal visible in the ciphertext as a whole.
Testing a possible Vigenère period with IC
Assume the ciphertext uses a repeating key but the period is unknown.
To test a candidate period of 5, divide the ciphertext into five streams:
If the true key period is 5, every character within Stream 1 was encrypted with the same key letter. The same is true independently for Streams 2 through 5.
Each stream is therefore one Caesar-shifted language sample, so its IC should tend toward the plaintext language’s IC when enough text is available.
If the candidate period is wrong, a stream mixes characters encrypted under different key letters. The mixed shifts flatten the distribution again, and its IC tends to remain lower.
A complete period test with real IC values
Consider an English sample containing 538 alphabetic characters. Before encryption, its measured raw IC is:
That value is somewhat above the often-quoted English average, which is useful in itself: real texts do not land on one exact universal number.
Encrypt the sample with the repeating five-letter key:
The resulting ciphertext begins:
The IC of the combined ciphertext falls to:
Now test possible periods. For each candidate, split the ciphertext into that many streams, calculate the IC of every stream, and take their average.
| Candidate period | Average stream IC | What it suggests |
|---|---|---|
| 1 | 0.0428 | Mixed polyalphabetic distribution |
| 2 | 0.0424 | No strong language-like recovery |
| 3 | 0.0432 | No strong language-like recovery |
| 4 | 0.0430 | No strong language-like recovery |
| 5 | 0.0760 | Strong candidate |
| 6 | 0.0431 | No strong language-like recovery |
| 7 | 0.0396 | No strong language-like recovery |
| 8 | 0.0423 | No strong language-like recovery |
| 9 | 0.0424 | No strong language-like recovery |
| 10 | 0.0781 | Also high because 10 is a multiple of the true period |
The jump at 5 is the important result.
With a candidate period of 5, each stream contains characters encrypted under one fixed key position. Their frequency distributions therefore recover the coincidence behavior of the underlying English sample.
The test has identified the true period used in this example without needing to know the letters O R B I T.
Why period 10 also produces a high IC
The value for 10 reveals an important feature that can otherwise look like a false result.
If the true period is 5, then positions:
all use the same key position.
Splitting by a candidate period of 10 divides that valid stream into two smaller streams:
Both still contain characters encrypted with the same shift.
A multiple of the true period can therefore produce high stream IC values as well.
The smallest convincing period is often more interesting than a larger multiple, provided other evidence supports it.
This is also why IC results should be interpreted as a pattern across candidate periods rather than as a contest where the single highest decimal value automatically wins.
Averaging the streams can hide useful detail
The average stream IC is convenient for comparing many candidate periods, but it should not be the only number inspected.
For candidate period 5 in the example above, the individual stream ICs are approximately:
They are not identical.
Each stream contains only around one fifth of the original text, and each samples different plaintext positions. Statistical variation is therefore expected.
A candidate period is more convincing when several streams independently show language-like unevenness, rather than when one unusually high stream pulls an otherwise weak average upward.
Short ciphertext can make IC unstable
Index of Coincidence is a statistical measurement. Like most measurements based on observed frequencies, it becomes more reliable as the amount of data increases.
Suppose a ciphertext contains 60 letters and you test a candidate period of 10.
An IC calculated from six letters can vary dramatically depending on a few accidental repetitions.
If two or three characters happen to be the same, the value can look unusually high even when the candidate period is wrong.
The opposite problem can occur as well. A correct period can produce streams whose observed ICs are temporarily low because the sample is too small to resemble long-run language frequencies.
| Situation | Effect on IC analysis |
|---|---|
| Long ciphertext | Each tested stream has more observations and usually produces a more stable estimate. |
| Short ciphertext | Random variation can create misleadingly high or low values. |
| Very long candidate period | Splits the ciphertext into many small, noisy streams. |
| Small candidate period | Produces larger streams, but may mix several key positions when the guess is wrong. |
Reporting an IC to four decimal places does not create four-decimal certainty. The underlying sample size still determines how much evidence the statistic carries.
Raw IC and normalized IC can look completely different
Two calculators can analyze the same text and report values such as:
and:
Both may be correct.
The first is the raw coincidence probability. Some treatments normalize that value by multiplying it by the alphabet size.
For a 26-letter alphabet:
So:
Under the normalized convention, a uniform random A–Z distribution is near 1.0 and ordinary English is often around 1.7.
| Reference | Raw scale | Normalized ×26 scale |
|---|---|---|
| Uniform A–Z distribution | ≈ 0.0385 | ≈ 1.00 |
| Typical English reference | ≈ 0.067 | ≈ 1.73 |
Before comparing your result with a book, website, or program, check which convention it uses.
A difference caused by normalization is not a disagreement about the ciphertext.
The expected IC depends on the language
The familiar English reference value comes from English letter frequencies. It is not a universal constant of meaningful text.
Other languages have different frequency distributions. Even within one language, different corpora produce somewhat different values.
This matters when analyzing Vigenère ciphertext whose underlying language is unknown.
If an analyst expects English but the plaintext is actually written in another language, comparing every stream rigidly against an English IC benchmark can lead to poor conclusions.
The alphabet matters too. The uniform baseline 1/26 applies specifically to an alphabet of 26 equally likely symbols.
With an alphabet of q symbols, the uniform baseline becomes:
The assumptions used to calculate and interpret IC should therefore match the actual cipher alphabet and likely plaintext language.
IC and Kasiski examination look for different evidence
Both methods can help investigate a repeating Vigenère period, but they do not ask the same question.
Kasiski examination looks for repeated ciphertext sequences and studies the distances between them.
Index of Coincidence looks at letter-distribution unevenness after the ciphertext has been grouped according to a candidate period.
| Method | Evidence used | Main limitation |
|---|---|---|
| Kasiski examination | Repeated ciphertext sequences and their spacing | Needs useful repetitions to appear. |
| Index of Coincidence | Statistical unevenness of letter frequencies | Needs enough characters for stable frequency estimates. |
Agreement between independent forms of evidence is more convincing than relying on one isolated signal.
Both methods fit within the broader process of breaking a repeating-key Vigenère cipher .
What the Index of Coincidence cannot tell you
A high IC is not a plaintext detector.
Monoalphabetic substitution preserves frequency unevenness, so a Caesar ciphertext can have essentially the same IC as the plaintext even though none of its letters are readable.
A high IC also does not identify which Caesar shift was used in a Vigenère stream.
Likewise, an IC spike at period 5 does not tell you that the key is ORBIT. It tells you that grouping the ciphertext every five positions produces distributions consistent with recurring single-shift streams.
Further analysis is still required to identify those shifts and recover the key.
Why the statistic works so well on Vigenère
The usefulness of IC comes from one property of substitution that is easy to overlook: a fixed alphabet shift changes letter identities without changing the shape of their frequency distribution.
Vigenère hides that shape by mixing several shifts together.
Guess the correct period, separate those shifts again, and the uneven language distributions reappear.
That is the Index of Coincidence’s real role in Vigenère cryptanalysis. It does not decipher the message by itself. It measures whether the way you have organized the ciphertext restores the statistical structure that a repeating-key cipher temporarily concealed.
