What is the cipher behind this circular cyphertext?

Question

I have a cipher and I am wondering if - provided you have both the cipher-text and the clear text, given below, can you describe the cipher which was used?

---

The cipher-text is in this image:

I know somebody will post this in a comment any second, so right away, here is some ASCII representation of the circular text.
However, I will not say that you can solve this cipher by this ASCII string alone. On the other hand, you will not need the digital form of that image, just what you are seeing. No dirty tricks played.

HA, TITEXGB   DTESTRI AT UE A ONTCTSOMNDEOTA DN UEYCN HRUSN.IETH AEPT CR  F TNGEYI?OI A RCA UODUTYE COHNTUI?EURYI HEPSA T R

---

This is the clear-text of the cipher:

YOU CAN READ THIS CIPHERTEXT? THAT IS GREAT, BUT IT DOES NOT MEAN YOU CAN UNDERSTAND THE CIPHER YET. CAN YOU FIGURE IT OUT?

Can you describe the cipher used in this puzzle ?

---

---

Hints

The following hints are all (valid) conclusions drawn from partial-solutions below or discussions in the comments. They are certainly not required and may spoil your fun (or influence you into a certain line of thinking), so I'm putting them into spoiler tags.

Hint #1:

The cipher does not alter any letters themselves. Number of each letter type is the same in clear-text and cipher-text.

Hint #2:

Both cipher- and clear-text are cyclic. There is no distinct starting point neither for encrypting, nor for decrypting. i.e. if "ABCDEFG" maps on "TUVWXYZ", then "DEFGABCD" will map onto "WXYZTUV" or "XYZTUVW" or any other cyclic permutation of the above. In all cases you can create the same "cycles" from clear-text and cipher-text.

Hint #3:

This should be obvious for a cipher, but with the cipher it is possible to encrypt/decrypt arbitrary messages reliably.

Hint #4:

The following pair of clean and cipher text was requested by Bobson and archaephyrryx.

THE QUICK BROWN FOX JUMPS OVER THE LAZY DOG, THEN THE LAZY DOG ROLLS UNDER THE QUICK BROWN FOX.WHO LIKES THIS DOG I WONDER?

becomes:

WITDOES E HNOCBI RKW TFQ  OURNJ EOMAWPE HXSUO,RV DL GT EHAEOZY HN DLG T DET OLLOE RZHNYCSBIRU TK.QF U HERON I ODWKLOH ISXG?

Hint #5:

You may want to consider the difference between mapping positions and moving or swapping characters in the context of a sequentially applied algorithm and reversibility of a cipher.

Hint #6 (last hint):

The cipher is a sequential algorithm treating 26 'classes' of letters in sequence (and leaving a 27th class untouched). Deciphering will require a reversal of both the sequence of the 26 steps as well as each of the individual 26 steps. The sequence follows the natural 'order' of the 26 classes.

Answer 1

2nd attempt - Trying to find a sequence using only the position of the characters has yielded no meaningful result. It appears that a value for each character is also being used. I think the cipher uses the position of each character in the clear text (from 1 to 123) minus its character value to determine its position in the cipher text. If the calculated position is less than 1, it is wrapped from the beginning to the end.

The following character values are the best that I have found so far. This is not the final answer but the results are closer than any of my previous efforts

Character   ?   ,   .   +   A   B   C   D   E   F   G   H   I   M   N   O   P   R   S   T   U   X   Y
Value       42  41  40  40  40  35  37  38  35  35  31  31  30  24  25  24  25  23  21  18  17  18  13

When the cipher text position is calculated from the clear text position minus the character value, the results are as follows (using "+" to help track the spaces):

My cipher text  H   A   ,   +   E   I   T   G   +   X   T   B   +   T   D   S   T   +   R   E   I   +   A   T   +   E   +   O   T   U   A   N   T   +   C   O   S   M   D   N   A   T   E   O   +   D   N   +   N   U   C   E   Y   +   H   R   N   U   E   S   .   +   I   H   A   E   T   +   C   P   T   +   R   F   +   +   N   E   ?   G   I   T   O   Y   +   I   A   +   C   R   U   Y   A   +   O   U   D   E   T   +   O   T   U   C   N   H   I   R   U   ?   E   Y   +   I   E   H   A   +   P   S   T   +   R
OP cipher text  H   A   ,   +   T   I   T   E   X   G   B   +   +   +   D   T   E   S   T   R   I   +   A   T   +   U   E   +   A   +   O   N   T   C   T   S   O   M   N   D   E   O   T   A   +   D   N   +   U   E   Y   C   N   +   H   R   U   S   N   .   I   E   T   H   +   A   E   P   T   +   C   R   +   +   F   +   T   N   G   E   Y   I   ?   O   I   +   A   +   R   C   A   +   U   O   D   U   T   Y   E   +   C   O   H   N   T   U   I   ?   E   U   R   Y   I   +   H   E   P   S   A   +   T   +   R

How three spaces come together in the OP's cipher text, is one peculiarity that I do not understand. I'll keep working on this but would be happy if somebody else found the correct solution.

Answer 2

Here are some thoughts I have before any extra info was revealed.

I believe the ciphertext is a permutation of the plaintext, treating spaces and symbols and letters alike. I also suspect the text is in Latin. (Edit: Never mind, BmyGuest has added that it's English.)

The letter frequencies of the text are similar to that of English but actually closer to that of Latin. Note in particular the increased frequency of U (which also serves the function of V in Latin), the relative rarity of H, the total lack of Latin non-letters J, K, W, and V. I can't be sure though because of the relatively small counts. I could also be another Romance language.

t : 14
e : 11
a : 8
u : 7
i : 7
n : 7
r : 6
o : 6
h : 5
c : 5
y : 4
d : 4
s : 4
g : 2
p : 2
x : 1
m : 1
f : 1
b : 1
z : 0
w : 0
v : 0
l : 0
k : 0
j : 0
q : 0

Of the 123 characters, there are 96 letters, there are also 23 spaces, two question marks, a period and a comma. This is consistent with a typical word length of about 4.2, totally reasonable. Likewise, three sentences averaging 32 letters each and 7-8 words each is reasonable, though on the short end for words per sentence. Two of the sentences are questions.

So, given that the character statistics are totally consistent that of plain text, I think the ciphertext is simply a rearrangement of the characters in the plaintext. Something that substituted for symbols for other symbols would not have produced these statistics. Moreover, there are no repeating substrings of length 3 or more, which further suggests rearrangement rather than substitution.

What rule could permute the letters in the ciphertext? My first thought was simply taking every $n$th letter, which would also explain why the message is in a circle, to wrap around. But trying this for all possible $n$ didn't give anything that struck me as intelligible.

Another thought is that each letter tells you to look at the next letter $k$ spaces forward, perhaps with A=1, ..., Z=26 and something for non-letters. But then one would likely get into a loop. Unless maybe you skip letters you've already used? I'll have to this more about whether this can be a reversible encoding.

Ideas to proceed:

• Simply find a Latin famous quote that could match. It should have two sentences and a question mark. Maybe a part of the Lorem Ipsum text?
• Look at the distribution of spaces to see what type of permutation might give reasonable word length
• Wait for more hints

Answer 3

Given that we know that the circular nature of the puzzle is important, and that there are an odd number of characters (123, specifically) it seems a pretty safe conclusion that we need to go around the circle at least twice to decode it, probably continuously.

---

A naive approach is "every other letter, going around twice". That produces

H,TTXB DETIA EAOTTONET NUYNHUNIT ETC FTGY?IARAUDTECHTIERIHPATR A IEG TSR TU NCSMDOAD EC RS.EHAP R NEIO C OUY ONU?UY ES

Clearly, that's not right - even rotating the starting point won't help, because that would just rotate the result, and there's no clear English in that result.

Trying 3 and 41 won't work, because they'll just loop over the same letters each time (3*41 = 123). Brute-forcing each possible skip number up to 123 confirms the naive approach doesn't work (although using 41 from the provided starting point does produce "HO?HO?HO?HO?HO?HO?HO?HO?HO?" over and over again, which is seasonally appropriate).

---

Now that we've concluded that the there isn't a consistent skip value, lets look at punctuation.

Of the 123 characters, there are 96 letters, there are also 23 spaces, two question marks, a period and a comma.

From each punctuation mark (as 0), spaces can be found in the following positions:

,

1, 9, 10, 11, 19, 22, 25, 27, 42, 45, 51, 62, 67, 70, 71, 73, 83, 85, 89, 97, 111, 117, 119

.

5, 10, 13, 14, 16, 26, 28, 32, 40, 54, 60, 62, 67, 75, 76, 77, 85, 88, 91, 93, 108, 111, 117

?

3, 5, 9, 17, 31, 37, 39, 44, 52, 53, 54, 62, 65, 68, 70, 85, 88, 94, 105, 110, 113, 114, 116

? #2

6, 12, 14, 19, 27, 28, 29, 37, 40, 43, 45, 60, 63, 69, 80, 85, 88, 89, 91, 101, 103, 107, 115

I'm not seeing anything obvious from that information, though. There's no numbers that show on all lists (which as we'd expect given the lack of a simple skip).

---

Update:

I've been playing with an extended version of this plan. Given the plaintext only has a single . in it (at position 60), I generated a list of every possible way to generate the sequence . CAN Y. A small sample of the output:

60,4,34,2,32,4,51
60,4,34,2,39,4,51
60,4,34,2,32,12,51
60,12,34,2,32,4,51
60,4,34,2,32,13,51
60,13,34,2,32,4,51
60,4,34,2,32,14,51
...
60,114,101,119,104,120,112
60,120,101,119,104,114,112
60,114,101,119,104,122,112
60,122,101,119,104,114,112
60,120,101,119,104,120,112
60,120,101,119,104,122,112
60,122,101,119,104,120,112
60,122,101,119,104,122,112

My goal was to look for patterns in the change between them, but there are 592480 possible combinations, which is a bit much to manually skim. If anyone's interested in trying to parse this data, I can upload it somewhere, but the file is over 12 MB worth of lines just like the ones above.

Answer 4

_{Posting another answer for a different approach.}

Based on xnor's analysis and some experiments I did based on the fact there is only a single ., I did a bit of analysis on the rare characters.

First, there's only two ? in the cyphertext, and one of them happens to be the same distance from the single . in both the plaintext and the cyphertext. I conclude that this is simply a coincidence, which means the other ? character maps to the ? at the end of the plaintext. So I changed the cypher/plain as follows:

HA, TITEXGB   DTESTRI AT UE A ONTCTSOMNDEOTA DN UEYCN HRUSN.IETH AEPT CR  F TNGEYI!OI A RCA UODUTYE COHNTUI?EURYI HEPSA T R
YOU CAN READ THIS CIPHERTEXT! THAT IS GREAT, BUT IT DOES NOT MEAN YOU CAN UNDERSTAND THE CIPHER YET. CAN YOU FIGURE IT OUT?

Note the ! used instead of the second ?. This lets us bring our "unique character" count to 7: X, M, F, B, ., ?, !

Some basic counting produces:

Char    Value   Cypher  Plain   Diff
X       88      8       26      18
M       77      37      61      24
F       70      74      109     35
B       66      10      45      35
.       46      59      99      40
?       63      107     122     15
!       33      82      28      69*

^{* This value is counted by moving in the same direction as the others, but it wraps around}

Also, for thoroughness' sake, here's the last two lines with the other ? turned into the !:

?       63      82      122     40 
!       33      107     28      44 

What does this mean? No idea yet.

---

Here's a more general breakdown (keeping the ?/! substitution). Top line is the position of the cyphered characters, bottom is the plaintext.

(Space)
  3, 11, 12, 13, 21, 24, 27, 29, 44, 47, 53, 64, 69, 72, 73, 75, 85, 87, 91, 99, 113, 119, 121
  3, 7, 12, 17, 29, 34, 37, 44, 48, 51, 56, 60, 65, 69, 73, 84, 88, 95, 100, 104, 108, 115, 118

!
  107
  28

,
  2
  43

.
  59
  99

?
  82
  122

A
  1, 22, 28, 43, 65, 86, 90, 118
  5, 10, 32, 41, 63, 71, 81, 102

B
  10
  45

C
  33, 51, 70, 89, 100
  4, 18, 70, 89, 101

D
  14, 39, 45, 94
  11, 52, 76, 83

E
  7, 16, 26, 40, 49, 61, 66, 79, 98, 108, 115
  9, 22, 25, 40, 54, 62, 77, 87, 93, 97, 114

F
  74
  109

G
  9, 78
  38, 111

H
  0, 54, 63, 102, 114
  14, 21, 31, 86, 92

I
  5, 20, 60, 81, 84, 106, 112
  15, 19, 35, 49, 90, 110, 116

M
  37
  61

N
  31, 38, 46, 52, 58, 77, 103
  6, 57, 64, 72, 75, 82, 103

O
  30, 36, 41, 83, 93, 101
  1, 53, 58, 67, 106, 119

P
  67, 116
  20, 91

R
  19, 55, 71, 88, 110, 122
  8, 23, 39, 78, 94, 113

S
  17, 35, 57, 117
  16, 36, 55, 79

T
  4, 6, 15, 18, 23, 32, 34, 42, 62, 68, 76, 96, 104, 120
  13, 24, 27, 30, 33, 42, 47, 50, 59, 80, 85, 98, 117, 121

U
  25, 48, 56, 92, 95, 105, 109
  2, 46, 68, 74, 107, 112, 120

X
  8
  26

Y
  50, 80, 97, 111
  0, 66, 96, 105

Answer 5

UPDATED

Until more hints are provided, the only thing I can really supply is a dump of all of the cyclic rotations of all of the co-prime sampling intervals of the ciphertext (and, separately, the plaintext). Each are 123*80 = 9840 lines. I hope that is useful to someone. I wrote the code in Haskell. The files are each 1MB, so I am splitting them up into several parts to fit it all on pastebin.

Cipher

Part 1: http://pastebin.com/mD2DuL5c

Part 2: http://pastebin.com/BPCNWwa3

Part 3: http://pastebin.com/XW61r97s

Plain

Part 1: http://pastebin.com/E2ef2bRL

Part 2: http://pastebin.com/mPhx0iLC

Part 3: http://pastebin.com/TCKhwtn6

EDIT

I have made no changes to the dumps above, but I just thought a bit more about the nature of this problem and came to a number of impressions/conclusions, which are not exactly rigorous but have some basis.

From the information provided, both in the post and in its comments, I have arrived at several conclusions:

Automorphism

Because the decryption does not rely on the orientation of the cyclical ciphertext or the starting point, we can consider this problem from a modular-arithmetic standpoint. We also know that all of the characters in the plaintext $p$ are preserved in the ciphertext $c$, merely subjected to a rearrangement. We can therefore treat the cipher as an automorphism, because it is a mapping of $\mathbb{N}_m \rightarrow \mathbb{N}_m$, where $m$ is the length of the message. We will therefore consider $E$ to be the encryption function, and $D$ to be the decryption function, where $D(E(p)) = p$. We also know that, for an arbitrary cyclic shift $C$, we have $E(C(p)) = C(E(p))$ and $D(C(c)) = C(D(c))$. We thus conclude that $E$ and $D$ are pure, stateless functions, meaning that they produce the same result for the same output independent of any history. We also know that $E$ and $D$ depend only on the positions of letters, and not on their values. For,if this were the case, the size of the domain of $E$ would be $m*L$, where the alphabet of the messages is of size $L$, while the range would still be $m$, as the letters themselves are merely transposed and not transformed; by the Pigeon-Hole principle, there must therefore be multiple index-value pairs that map to the same index of the ciphertext, which we know not to be the case. We therefore consider $e$ instead of $E$, which is the function mapping an index in plaintext to the ciphertext index when $E$ is applied, which we state as $$p_i = c_{e(i)}\;\forall\;1 \le i \le m$$

We therefore only need to determine the mapping of $e$, which we know must be linear if $E(C(P)) = C(E(P)) \implies e(i+x) = e(i)+x$. After I create a table of the positions of each letter in ciphertext and plaintext, I will post the results of that.

Results

(I know someone else has posted these, but this is in a slightly different form, and it doesn't hurt)

[3,7,12,17,29,34,37,44,48,51,56,60,65,69,73,84,88,95,100,104,108,115,118] -> [3,11,12,13,21,24,27,29,44,47,53,64,69,72,73,75,85,87,91,99,113,119,121]
[43] -> [2]
[99] -> [59]
[28,122] -> [82,107]
[5,10,32,41,63,71,81,102] -> [1,22,28,43,65,86,90,118]
[45] -> [10]
[4,18,70,89,101] -> [33,51,70,89,100]
[11,52,76,83] -> [14,39,45,94]
[9,22,25,40,54,62,77,87,93,97,114] -> [7,16,26,40,49,61,66,79,98,108,115]
[109] -> [74]
[38,111] -> [9,78]
[14,21,31,86,92] -> [0,54,63,102,114]
[15,19,35,49,90,110,116] -> [5,20,60,81,84,106,112]
[61] -> [37]
[6,57,64,72,75,82,103] -> [31,38,46,52,58,77,103]
[1,53,58,67,106,119] -> [30,36,41,83,93,101]
[20,91] -> [67,116]
[8,23,39,78,94,113] -> [19,55,71,88,110,122]
[16,36,55,79] -> [17,35,57,117]
[13,24,27,30,33,42,47,50,59,80,85,98,117,121] -> [4,6,15,18,23,32,34,42,62,68,76,96,104,120]
[2,46,68,74,107,112,120] -> [25,48,56,92,95,105,109]
[26] -> [8]
[0,66,96,105] -> [50,80,97,111]

Answer 6

As this puzzle has been open for a while, I'm now going to post the solution as intended. At this time, puzzlers on this site have 99% solved the cipher, and it is mainly technical detail which remains to be sorted - which might not be much fun. So, as a wrap-up here is the cipher as used in this puzzle. Do not read on, if you still want to find it by yourself.

---

The cipher is a simple transposition cipher acting

on letters only.
Each letter is shifted X positions to its right/clockwise, where X stand for the value of the letter (A=1, B=2, ...).
Spaces, numbers, dots etc. remain unaffected.
All characters (including spaces) count for the shifting purspose.

A few complications come in to have it a reversible cipher:

The encoding process is sequentially done for the letters
The 'simultaneously' .
First all A's are shifted "simultaneously" by 1 position to the right.
next all B's are shifted "simultaneously" by 2 positions to the right.
next all C's are shifted "simultaneously" by 3 positions to the right.
etc.
"simultaneously" shifting is necessary, to avoid letters of the same type 'overtaking' each other, which would destroy reversibility of the cipher.

In order to decode a cipher text

Reverse the process!.
First all Z's are shifted "simultaneously" by 26 position to the left.
next all Y's are shifted "simultaneously" by 25 positions to the left.
next all X's are shifted "simultaneously" by 24 positions to the left.
etc.

Here is the example encoding process:

Answer 7

Work in progress:

In the quick brown fox example,

T is in position 1 in the clear. T is the 20th letter of the alphabet. In the cipher text, there's a T at position 1 + 20 + 1.

H is in position 2 in the clear. H is the 8th letter of the alphabet. In the cipher text, there's an H at position 2 + 8 + 1.

E is in position 3 in the clear. E is the 5th letter of the alphabet. In the cipher text, there's an E at position 3 + 5 + 1.

Q is in position 5 in the clear (4 is a space). Q is the 17th letter of the alphabet. In the cipher text, there's a Q at position 5 + 17 + ... 2. So it's not perfect.

Continuing, U is in position 6 in the clear. U is the 21st letter of the alphabet. In the cipher text, there's a U at position 6 + 21 + ... 0. Again, not perfect but eerily close.

I: 7 + 9 + 0.

C: 8 + 3 + 3

K: 9 + 11 + -1

There's something like a pattern here or else it's just coincidence that these letters are found shifted ahead by approximately their alphabetic position.

Edit: I was just thinking, if I shift a letter, say T, ahead by 20 then the 20 characters skipped over are effectively shifted left by 1. So every time a letter is jumped over by another letter it is displaced backward. I'm also thinking that if every letter gets moved forward by its alphabetic position then As shift few letters while Zs shift many. All else being equal, a letter 20 spaces ahead of a cluster of As will move less than one 20 spaces ahead of some Zs. This may account for the variability.

Answer 8

Well, here is what I've got. I think the algorithm is almost clear, but something is off and I can't figure out why; since I probably won't have time to keep trying in the next days, I hope someone else will fix the missing parts.

The cipher's logic is this:

for each letter X from A to Z, starting from the end of the string, take every X and move it forward of n positions, where n is the position of X in the alphabet.

A key factor for reversibility, with this algorithm, is that one has to be able to execute the same steps backwards. To do so, one has to know at each step

which one is the first X that has been moved. It's for this reason, I suspect, that before moving the Xs you have to take the first n letters of the string and move them to the end, and put them back after the movements.

This is what I obtained from the quick brown fox thing, using this algorithm:

THE QUICK BROWN FOX JUMPS OVER THE LZaY DOG, THEN THE LZaY DOG ROLLS UNDER THE QUICK BROWN FOX.WHO LIKES THIS DOG I WONDER?

...

tdoes Ue hnocbWi rkX tUfq Vor nj oeZmaYpe osh,r dl gt eZhaYeo hn dlg Ut det ollUoe rhncsbWir tkX.qWf  heron i odkloh Wisg?i

...

witdoes e hnocbi rkw tfq our nj oemawpe oxush,rv dl gt ehaeozy hn dlg t det olloe rzhnycsbiru tk.qf u heron i odwkloh isxg?

Which is almost right. But the saddest part is, I can't even decode it correctly. The result of the decoding is something like:

TEHUQ IKC BROWN FOX JUMPSO VER THE LAZY DOG, THEN THE LAZY DOG ROLLS UNDER THE QUICKB RWONF XO.WHO LKEIS THIS DOG I WODENR?

So, to sum up: I think this is the right path and we're almost there, PLEASE someone fix this.