How to Break the Vigenère Cipher

The traditional repeating-key Vigenère cipher can be attacked without being given its key. The important weakness is not the Caesar-style shift used at each position, but the fact that a short key is repeated throughout the message.

Once the length of that repeating key can be estimated, the ciphertext can be separated into groups according to their key positions. Each group has been encrypted with the same shift, making the groups much more similar to Caesar-cipher ciphertext than to a full polyalphabetic message. Frequency analysis can then be used to estimate the individual shifts and reconstruct a candidate key.

This is the basic strategy behind classical Vigenère cryptanalysis:

The process is different from ordinary Vigenère decryption. If you already know the key, decryption is a direct mathematical operation. If the key is unknown, the task is to infer information about the key from patterns in the ciphertext.

For the mechanics of ordinary encryption and decryption, see How to Encrypt and Decrypt With the Vigenère Cipher . This article concentrates on the cryptanalytic problem.

Why the repeating-key Vigenère cipher can be broken

The traditional Vigenère cipher changes its substitution according to the key.

If the key is:

the shifts repeat:

That repetition creates a period of five. The first, sixth, eleventh, and sixteenth alphabetic positions all use the shift associated with L. The second, seventh, twelfth, and seventeenth positions all use the shift associated with E. The same relationship continues through the key.

This means the ciphertext is not one completely unrelated sequence of substitutions. It is an interleaving of several fixed-shift streams.

If the key length is five, the ciphertext can conceptually be divided into five streams:

Every stream uses one fixed Caesar-style shift. That is the structural weakness an attacker is trying to expose.

The University of Rhode Island describes this same approach: once the keyword length is known, the ciphertext can be divided into that many groups and each group can be attacked with frequency analysis.

The first problem: finding the key length

Knowing that the key repeats is not enough.

Suppose the ciphertext is:

You still do not know whether the key has length 3, 5, 7, 10, or some other value. Finding the key length is therefore usually the first major stage of the attack.

Two classical approaches are particularly useful:

• Kasiski examination, which looks for repeated ciphertext sequences and the distances between them;
• Index of Coincidence, which measures how often letters coincide and can help identify likely periods.

A Friedman-style analysis is another statistical approach to estimating the likely key length. The University of Rhode Island treatment presents both Friedman and Kasiski methods as ways of determining the keyword length before attacking the individual streams.

Neither method should be treated as an infallible one-number answer. Short ciphertexts, accidental repetitions, unusual plaintext, and coincidental statistical patterns can all produce misleading candidates.

The practical goal is therefore to generate plausible key lengths, not to assume that the first promising number must be correct.

Kasiski examination

The Kasiski examination uses repeated sequences in the ciphertext to look for evidence of the key’s period.

The reasoning is straightforward.

Suppose the same sequence of plaintext letters appears twice and those occurrences happen to align with the same positions of the repeating key. Because both plaintext sequence and key sequence are the same, the resulting ciphertext sequence can also repeat.

If that repeated ciphertext sequence occurs at two positions separated by a distance that is a multiple of the key length, the distance contains information about the period.

For example, suppose a repeated ciphertext sequence appears at positions:

The distance is:

Possible key lengths could therefore include factors of 30:

A single repeated sequence does not prove that any one of those values is the key length. The useful evidence comes from multiple repeated sequences.

Suppose other repeated groups produce distances such as:

A common factor of those distances may point toward a likely period. The University of Rhode Island gives the same principle: repeated ciphertext groups can occur when plaintext sequences align with the repeating keyword, and the distances between those repetitions can reveal multiples of the keyword length.

Why repeated sequences are only clues

A repeated ciphertext sequence does not automatically mean that the key length divides its distance. The same letters can occur by coincidence.

For example, a short sequence such as:

can appear naturally in a sufficiently long ciphertext even when the occurrences have nothing to do with the key period.

This is why longer repeated sequences are generally more informative than very short ones, and why several independent distances are more useful than one.

Kasiski examination therefore works best as a candidate-generation method. It tells you which periods deserve further testing. It does not finish the cryptanalysis by itself.

Index of Coincidence

The Index of Coincidence, often abbreviated IoC, approaches the key-length problem statistically rather than by searching specifically for repeated strings.

It measures how likely two randomly selected letters from a text are to be the same.

For a ciphertext containing letter counts nA,nB,…,nZ and a total of N letters, the usual Index of Coincidence calculation is:

The value provides information about the statistical structure of the text.

Natural English has a much more uneven letter distribution than uniformly random text. A monoalphabetic substitution preserves those frequency relationships, while a polyalphabetic cipher such as Vigenère distributes them across multiple shifts. The University of Rhode Island uses coincidence measurements as one way to distinguish likely cipher structures and estimate a Vigenère keyword length.

The important point is that IoC is not simply asking:

> “What is the most common letter?”

It asks about the overall probability that two selected letters match.

That makes it useful for detecting statistical structure that ordinary frequency counts alone may not reveal.

Using candidate key lengths

Suppose the ciphertext analysis produces these possible periods:

The next step is not to choose one arbitrarily. Instead, test each candidate against the ciphertext.

If the true key length is five, then dividing the ciphertext into five positional streams should produce groups that each resemble Caesar-shifted English more closely than an incorrect grouping does.

The same principle works in reverse: an incorrect key length tends to mix positions that were encrypted with different shifts, making the frequency distribution less coherent.

This gives a practical way to compare candidate periods.

For each candidate k:

1. Take every kth ciphertext letter.
2. Create k separate streams.
3. Measure the statistical properties of those streams.
4. Look for a grouping consistent with shifted English.

Emory University’s Vigenère materials demonstrate this exact type of analysis by extracting every kth letter for a presumed key length and comparing the resulting frequency distribution with rotated English-frequency distributions.

Turning the ciphertext into Caesar-like streams

Once a plausible key length has been selected, the ciphertext can be reorganized.

Suppose the suspected key length is four.

The ciphertext is divided into four streams:

Why does this help?

Because every position in a particular stream was encrypted using the same key letter.

If the first key letter is D, every letter in Stream 1 was shifted by 3.

If the second key letter is Q, every letter in Stream 2 was shifted by 16.

And so on.

Each stream can therefore be treated approximately as a Caesar cipher. The original Vigenère problem has been reduced to several smaller shift-cipher problems.

This is the most important structural transition in the attack.

Frequency analysis of each stream

After separating the ciphertext into streams, the next task is to determine the shift used for each stream.

Suppose one stream contains:

The letters do not look like ordinary English because the stream is still shifted.

But if the same Caesar shift has been applied to every letter, the relative frequency pattern remains recognizable.

For English plaintext, letters such as E, T, A, O, and I occur more often than many others.

After encryption with one fixed shift, the identities of those high-frequency letters change, but their relative concentration is still present.

This allows the analyst to compare the observed stream against expected English frequencies under each of the 26 possible shifts.

Emory’s treatment demonstrates this approach by constructing shifted versions of English frequency distributions and comparing them with the frequency distribution of a Vigenère subsequence.

Testing the 26 possible shifts

Each stream has only 26 possible Caesar shifts under the standard A-Z Vigenère model.

For one stream, you can therefore test:

For each candidate shift, decrypt the stream and ask whether its letter distribution resembles English.

A simple statistical comparison can rank the candidates.

For example, if the observed stream has a frequency distribution u, and vᵢ represents the expected English distribution rotated by shift i, one approach is to compare their similarity using a dot product:

A larger value indicates greater alignment between the two distributions under that shift. Emory’s worked Vigenère analysis uses this type of frequency-vector comparison to identify the likely Caesar shift for a ciphertext subsequence.

Other implementations use measures such as chi-squared distance instead. The underlying goal is the same: rank the 26 possible shifts by how well their decrypted statistics fit the expected language.

Recovering the key one letter at a time

Once the likely shift for each stream has been estimated, those shifts can be converted back into key letters.

Using the conventional mapping:

a shift of:

corresponds to A.

A shift of:

corresponds to B.

A shift of:

corresponds to C.

And so on.

If the four streams produce estimated shifts:

the corresponding candidate key is:

The exact key cannot be assumed correct merely because each individual stream produced a plausible statistical match. The complete decrypted message still needs to be examined.

Why frequency analysis can fail on short ciphertext

Frequency analysis is statistical. That means it needs enough data.

If a stream contains only five or ten letters, the observed frequencies can differ dramatically from the frequencies expected in ordinary English simply because the sample is too small.

For example, a short piece of English could contain no E at all. That does not mean E is absent from English. It means the sample is too small to represent the language reliably.

The same problem affects each Vigenère stream.

A long ciphertext with a five-letter key gives many letters to each stream. A short ciphertext with a ten-letter key gives relatively few letters to each stream.

This is why key length and ciphertext length cannot be considered independently. A longer key means fewer observations per key position for a fixed ciphertext length.

The University of Rhode Island explicitly notes that frequency analysis after splitting the ciphertext depends on having a sufficiently long ciphertext sample.

What if two key lengths both look plausible?

This is common.

Suppose the analysis suggests:

The larger values may simply be multiples of a shorter true period.

For example, if the true key length is five, then ten and fifteen also divide the ciphertext into groups that preserve the same underlying five-position structure.

This is why candidate key lengths should be reduced to their simplest plausible period when the evidence supports it.

But that reduction should not be automatic. A key can genuinely have a longer period, and an apparent shorter period may arise from repeated structure inside the key itself or from statistical coincidence.

The correct approach is to test the candidates against the resulting plaintext rather than relying solely on arithmetic factors.

Recovering the plaintext

Once a candidate key has been reconstructed, decrypt the entire ciphertext with that key.

At this point the analysis changes from statistical inference to direct verification.

A plausible key should produce plaintext with recognizable structure:

• Natural word boundaries
• Reasonable letter sequences
• Consistent spelling
• Coherent sentences

If the result contains mostly random-looking letters, the candidate key is probably wrong, incomplete, or based on an incorrect key-length assumption.

This is why cryptanalysis should not stop when a frequency test produces the highest score.

A statistical ranking identifies a candidate. The decrypted message determines whether that candidate actually makes sense.

Verifying a candidate key

A good candidate key should survive several checks.

First, decrypt the complete ciphertext. Second, examine the beginning, middle, and end rather than recognizing one short phrase and stopping. Third, check whether the plaintext has consistent language structure. Fourth, if the key appears to be a meaningful word, verify that the same key reproduces the observed ciphertext when the recovered plaintext is encrypted again.

That final round trip provides a particularly strong implementation check:

If re-encryption does not reproduce the ciphertext under the same character-handling rules, something is wrong with the candidate key, alignment, or implementation.

When Kasiski and Index of Coincidence disagree

The two methods do not have to produce the same first choice.

Kasiski examination depends on repeated sequences. If the plaintext does not contain useful repeated fragments, the evidence may be weak.

Index of Coincidence depends on statistical distributions. If the ciphertext is short, its estimates may also be noisy.

They are therefore complementary rather than interchangeable.

A practical analysis can use Kasiski examination to generate likely periods and IoC measurements to evaluate them.

For example:

The University of Rhode Island presents Kasiski and Friedman analysis as separate approaches to the key-length problem, while Emory’s material shows how a presumed key length can then be tested through subsequence frequency analysis.

A small conceptual example

Suppose a ciphertext is believed to use a four-letter key.

You divide it into four streams:

Now suppose statistical analysis suggests these shifts:

Convert the shifts to letters:

Candidate key:

You then decrypt the entire ciphertext with CODE. If the result is coherent English, the candidate gains strong support. If it is not, test another candidate shift or reconsider the assumed key length.

The example illustrates the workflow without implying that every ciphertext will produce such clean statistical evidence.

What makes a Vigenère ciphertext easier or harder to analyze?

Several factors affect the difficulty of classical cryptanalysis.

Ciphertext length

More ciphertext generally provides more statistical evidence. A very short ciphertext may not contain enough information to distinguish competing key lengths or shifts reliably.

Key length

A short repeating key creates fewer streams and gives each stream more observations. A longer key creates more streams, leaving fewer letters in each stream for the same total ciphertext length.

Plaintext language

Frequency-based methods work best when the plaintext has reasonably predictable statistical characteristics. Natural English provides useful letter-frequency information. Other languages have different distributions.

Repeated plaintext structure

Repeated words or phrases can produce repeated ciphertext sequences when their positions align with the key period, making Kasiski-style analysis more informative.

Character conventions

Spaces, punctuation, numbers, and other characters can affect positional alignment if an implementation treats them differently. When reproducing an analysis, the exact character-handling convention must therefore match the encryption method.

Can a Vigenère cipher always be broken?

No.

The statement needs qualification.

The traditional repeating-key Vigenère cipher has a structural weakness because the key repeats. With sufficiently long ciphertext and a suitably short repeating key, classical cryptanalysis can often exploit that periodic structure.

But a short ciphertext may not contain enough statistical evidence.

And if the key is as long as the message and used only once, the resulting system is no longer the ordinary short repeating-key Vigenère situation that these attacks target.

The important distinction is therefore between the traditional repeating-key construction and other systems that may use Vigenère-like operations with different keying arrangements.

Classical cryptanalysis resources specifically emphasize the relationship between key length, ciphertext amount, and the feasibility of frequency-based attacks.

Why breaking Vigenère is different from brute force

There are 26 possible Caesar shifts for an individual key position, but that does not mean the entire Vigenère key can simply be tested as 26 possibilities.

A key of length five has:

possible keys.

A key of length ten has:

possible combinations.

Classical cryptanalysis reduces this search dramatically by exploiting the cipher’s structure.

Instead of guessing the entire key at once, the analyst can:

That is fundamentally different from blindly trying every possible keyword.

The role of modern automated tools

Software can automate almost every statistical step of the process. A Vigenère cryptanalysis program can:

• search for repeated ciphertext sequences;
• calculate candidate key lengths;
• calculate Index of Coincidence values;
• divide ciphertext into positional streams;
• compare frequency distributions;
• rank candidate shifts;
• assemble candidate keys; and
• score resulting plaintext.

Emory University provides a Vigenère subsequence analyzer specifically for examining every kth letter of ciphertext under a presumed key length and comparing its frequencies with rotated English frequencies.

Automating these calculations does not change the underlying cryptanalysis.

The software is applying the same observations:

The complete Vigenère cryptanalysis workflow

The full process can now be summarized without reducing it to a single trick.

Start with ciphertext for which the repeating key is unknown. Look for repeated sequences and measure their distances. Use those distances to generate candidate key lengths. Calculate statistical measures such as Index of Coincidence to evaluate likely periods. For each promising key length, divide the ciphertext into positional streams. Treat each stream as a Caesar-like substitution. Compare its letter distribution against expected language frequencies. Select candidate shifts for each key position. Assemble those shifts into a candidate key. Decrypt the entire ciphertext. Read the resulting plaintext critically rather than accepting the highest statistical score automatically. Finally, re-encrypt the recovered plaintext with the candidate key and verify that it reproduces the original ciphertext under the same conventions.

That is the classical attack against the traditional repeating-key Vigenère cipher.

Why the attack works

The attack succeeds because the repeating key leaves a measurable structure in the ciphertext.

At the surface level, Vigenère appears to use many substitution alphabets. At the key-period level, however, each position in the cycle uses one fixed shift.

Once those positions can be separated, the polyalphabetic system becomes a collection of simpler substitution problems.

That is the weakness that Kasiski examination and coincidence-based methods help expose.

The encryption formula itself has not been “reversed” by some special mathematical shortcut. Instead, the attacker has inferred the structure of the key from statistical evidence and then solved the resulting smaller problems.

Continue learning about the Vigenère cipher

If you want to understand the encryption and decryption operations that this attack eventually reverses, read How to Encrypt and Decrypt With the Vigenère Cipher .

If you want the underlying mathematical mechanics in greater detail, see How Does the Vigenère Cipher Work? .

For the historical development of the cipher and the cryptographic systems that preceded it, continue to History of the Vigenère Cipher .

If you simply have a known key and want to decode a message, use the Vigenère Cipher Decoder and Encoder on the homepage instead of performing cryptanalysis manually.