Megan & the numeric cipher (693264475 etc)

Question

The other day I was chatting with Megan in Caffè Claudius when she said "You're good at maths, aren't you -- know anything about codes?"

Hm. Truth to say, I know a bit about some of them but am far from being an expert in solving them.

"The OH's got another obsession. This time it's codes. Last few days, he's had his nose in books, studying the literature. Yesterday he e-mailed one to me and wondered if I could solve it. Me?" She rummaged in her large canvas shoulder-bag which she'd put on the floor, then handed me a sheet of paper. [Names changed to protect the guilty.]

From: RD_Jones <[email protected]> To: Megan <[email protected]> Subject: A cipher puzzle for you

Hi Megan,
Here's a text which I've enciphered. I wonder if you can decipher it. The method I used to encipher it is one I found on the web, but the key is my own.
The plaintext is an extract from a published English-language work which can be found on the web. I've also put in the names of the work and the author. I based part of the key on the author's name, too. Then, when I was choosing how to fill out the rest of the key, I did that in the straightforward way.
I've ignored punctuation, and collapsed upper-case letters to lower-case. Thought I'd make it a bit easier to crack this cipher, so I've enciphered all the spaces. I hope the method's easy enough that you'll get clues from elementary cipher-breaking techniques.
6932644755 6919561446 6575417949 5683785753 6445693737 8755636657 5632466845 7263796956 9637552366 6557969636 6561186258 7378704953 7047851586 9464638125 7047575425 6366572696 3869569457 2175561784 7263666551 5671787963 8732178205 6412393783 7375567783 9738795569 6375569419 5721755757 3666257367 3665696375 5617841757 9516695694 5233624956 8783645640 5673636679 5368570634 7252636245 1575797816 6654521666 2561637895 7263796959 7322565781 0567273641 6545166951 5612186257 9367626637 9557535721 7975366575 1639569457 5739946620 5037359356 6379567783
6737545793 5636670475 7956366575 3737969516 8786381665 7548737863 7963475556 3565170451 5757917879 5368517579 3666375696 4466795518 8737579364 4951755635 7217557935 6932644757 5587378633 7375568187 3279634755 7969637557 5739946656 3667978737 5633665636 6793564056 4375795636 6796364179 4579693736 5697975572 1755737979 4782056366 4746726381 6124556937 2536654178 7969593503 7356266372 5796917955 6351756249 5503735754 4564059417 8572179753 6656379563 7556637957 8412205963 6868638732 7957935836 6757978738 7951575478 6347553685 6366684784 6684755418
6959467466 9466795736 7366563797 5567784948 4757537851 6695418695 7563646724 5636656379 7542685563 6851687947 8593636665 5753536645 7563646720 5626638627 5537379512 2579694584 6679781256 3666847846 6847551669 5677847546 6797553664 7551739634 6684572637 9695796945 7579178796 3666567363 6679516695 7969458366 8273756336 6536645641 0567783973 8451575791 7879263666 5579693736 5695673757 5636120515 6447847978 6386337375 5468684879 5566372563 7957217556 6379578412 2059636868 6387327956 1051665636 6756748796 3366536857 9694565783 3704561479 7244665037
3785246879 5683784686 3666547851 6695796973 6461579356 8442575737 8457969179 5037359639 5663795677 8367375457 9356366704 7579503737 8575641225 8167637912 5636657969 4565329568 6342975556 3575445663 5836666487 9633665570 4780526362 4205663795 6173795635 8166571736 3862205756 9372503735 1582375458 3666648796 3366556947 8451784579 6945646375 7563666552 6366627553 6857969457 0478057563 6467245869 1636655771 5503735691 9586912625 6147972446 6503737852 4687956863 6665478516 6957969736 4615726946 6503735784 7973786649 5687836457 9694582736 1521757956
6636569795 5772550373 5677379586 9126257969 4784572694 6650373567 2105616322 6317897557 9357579419 0579694587 3455773550 3735664704 7856721056 1632263178 9755474846 7795726379 6957969737 8757936655 7745503735 7932956445 6837378572 4462755165 3579691795 7969737875 7936656919 5166536779 6336653665 7536445753 7379695168 7863816656 7783674787 9057269638 6957237329 5474676378 4563665156 4366796951 6695726963 8695694594 7563784950 3735793575 6917845726 3796956963 6455775550 3737858694 8625613362 5637552386 2495636656 4059781724 7851669503 7356917045
6637951756 2495683785 7969456240 5577655037 3593566379 5677836737 5457935636 6704757950 3737856436 6405636657 9696375564 1666647855 6937251617 5737892057 5636467245 5635878634 9557173637 9457535575 1639569451 5263797924 5664797924 9554704780 5677836124 6456148364 4755704780 5869632963 7569572694 6653668456 3795637554 7467216366 4957935037 3556878364 5796945197 0466797378 4536857969 4591668636 6655644665 6105756378 5178796973 7858366166 5930245

She went on. "I hope it really is simple. Can't see any key anyway."

I looked at it awhile. Even I could see that the figures didn't look random -- but they still looked mysterious. I leafed through to the end of the last page of Megan's printout. "Is this all there is?"

"All there is? There's enough of it! Yes, that's exactly what was in the e-mail."

"That's odd, then... No matter." Referring back to the start of the e-mail, "He says the key's partly based on the author's name."

"Well that's as helpful as locking your car keys in your car."

"What was he studying?"

"Books with codes in. Don't know exactly."

Can you folks solve it? Find the method and key, and identify the plaintext -- no need to copy the whole plaintext out. And if you do manage to crack it, it'd be nice if you could show the method you used to do that.

I hope the method can (eventually) be cracked and that there is enough ciphertext for your cryptanalysis to yield clues. If not, I'll have to get Megan to prevail upon her OH to try something easier next time!

---

Hint 1:

How many digits represent each character? One? Then only ten characters could be encoded, so not enough. Two? But the ciphertext length is odd. Three? How long a string can you find which occurs twice at two places which are not a multiple of 3 apart? Barring very improbable coincidences, what does that suggest, about how separate occurrences of the same string are enciphered, and about how each letter is enciphered?

Answer 1

The cipher is

a thing I have seen called a "straddling checkerboard" as described e.g. in this Wikipedia article. The order of "least significant" digits is 1859302467; the key is "ac doyle"; note that 1859 is his year of birth and (coincidentally?) 30 the last 2 digits of his year of death.

The text is

an extract from "The Dancing Men", plus a little coda saying what it is and who it's by.

Although punctuation has been removed,

digits have been left as themselves and I think bullet points or # signs or something have been turned into "z"s where a numbered list occurs.

How the cryptography was done:

The analysis below (left unaltered from when I wrote it apart from a remark on the fate of some conjectures I made) makes it fairly clear that we're dealing with something close to a simple substitution cipher with a variable-length encoding, probably always 1 or 2 digits.

Phlarx's answer then gets the structure of that cipher pretty much on the nose.

He overlooked the need for something to represent a space. That clearly has to be 5, 6, or 7 on frequency grounds. I wrote a little program to group up the digits and replace them with (wrong) letters and spaces, trying each of {5,6,7} as space and the other two as first digits of 2-digit numbers; Phlarx was absolutely right that 6,7 were the prefixes, and 5 is space.

To work out the correspondence between numbers and letters I confess I just used quipqiup; a little fixing up was necessary because of the issue with plaintext digits mentioned above. And then

looking at the letters represented by single digits it was obvious (even had the puzzle not said so explicitly) that Doyle's name was involved, and then considering the ordering of those letters (and the other letters which are alphabetical) led immediately to the structure of the cipher. I wouldn't have known its name, except that when thinking about this puzzle earlier I spent a few minutes looking at the results of a web search for something like "digit cipher" and one of them was the Wikipedia page linked above.

---

A remark on reputation

It seems to me that Phlarx deserves a large fraction of the credit for this solution, having identified essentially the Right Thing but just not done all the busywork to find the plaintext. Therefore:

• Anyone inclined to upvote this, please upvote Phlarx instead or as well.
• If once the dust has settled it looks to me like Phlarx has been shortchanged, I will use the bounty mechanism to fix that up as best I can.

---

[Here is what I had here before the actual solution above.]

Some uninspired notes in the hope that they may help someone. No spoilers because I certainly haven't solved anything. This may turn into an answer if inspiration strikes but certainly isn't one yet.

There are 2907 digits. That's 3x3x17x19. I suspect they don't want grouping into constant-size groups, not only because that factorization is unhelpful but also because ...

The distribution of digits is very uneven. In particular there are very few 0s. This could happen if the digit-string we've got is obtained by cramming together lots of numbers (because the first digit of each number is constrained not to be 0 if the number is more than one digit). There are 65 0s, suggesting about 650 possibly-0 digits, suggesting that a large majority of the digits here are first digits of >1-digit numbers, which is impossible. So there is probably some further thing at work making there be few zeros.

There are also a lot of 5s, 6s and 7s and a fair bit of variability beyond that.

Actual digit frequencies, 0 to 9: 65 127 125 325 216 515 598 487 178 271.

Frequencies of digit pairs (note: over all pairs, including overlapping ones; don't overinterpret), first indicated by row, second by column:

   -0  -1  -2  -3  -4  -5  -6  -7  -8  -9
0-  0   0   1  20  15  28   1   0   0   0
1-  6   1  14   1   4  15  34  40   6   6
2- 10  14   8   6  22  23  28   9   0   5
3-  1   2  14  11   8  40 108 115  17   9
4-  6  16   5   3  17  60  39  48   9  13
5- 20  40  11  27  24  70 150 135  20  14
6-  0  23  23 128  48  58 117  52  40  89
7- 16   4  40  77   8 106   4  18  85 129
8-  4  11   5  22  32  27  35  37   1   4
9-  2  16   4  30  38  85  61  33   0   2

The variation in frequency of digits, of pairs of digits, and also of longer sequences (see below) suggests that this cipher isn't mashing things up too much; perhaps it is a (homophonic?) substitution cipher mapping plaintext characters to (variable-length) sequences of digits.

Spaces are included. I think something like 20% of typical English text is spaces, so if we guess that we have roughly twice as many ciphertext digits as plaintext characters then perhaps there are about 300 spaces. If spaces are indicated by some single consistent sequence of digits (which isn't at all a given) then the pair frequencies suggest it must be a single digit representing space. All digits other than 0 (which is surely too rare to indicate a space) occur doubled at least once, but maybe a double space indicates a paragraph break. Or just a double space. Or a space adjacent to a letter whose representation (one of whose representations?) starts or ends with the digit that denotes a space.

Some sequences of digits occur rather frequently. Here are some. Each line indicates number of occurrences followed by the actual sequence.

32 6366
35 7969
20 0373
24 3665
22 6379
25 5636
25 5796
20 5037
25 57969
15 03735
19 56366
13 79694
12 96945
20 50373
13 579694
 8 563665
 8 663795
12 796945
15 503735
 9 516695
 8 636665
 7 5663795
12 5796945
 6 5503735
 6 5037356
 5 50373785
 5 26379695
 4 78516695
 4 56778367
 4 95796973
 4 75636467
 4 85796945
 4 57563646
 4 56364672
 5 72637969
 5 57263796
 4 57969458
 4 46650373
 4 57969456
 4 79633665
 4 56637956
 4 756364672
 5 726379695
 4 5756364672
 5 5726379695

Quite a few of these start and end with 5; I would be unsurprised if it turned out that 5 denotes a space. (But for sure not all 5s indicate spaces.) I suspect that either 6366 or 7969 is TH and that we aren't seeing HE among the common 4-digit sequences because E happens to be a single letter. If so, looking at digit-pair frequencies I suspect it's 7969 that's TH. But this is all as yet the purest guesswork. [EDITED to add: actually 6366 was IN and 7969 was TH. And yes, E was a single digit, which is what I meant when I wrote "single letter" above.]

Further diddling with digit and pair counts and common sequences gives me a different suspicion incompatible with the one in the previous paragraph, namely that the frequently occurring 503735 is the, splitting as 5/0/37/3/5. (The frequency of 37 is about right for H.) But we then encounter a difficulty: there are also lots of 50373785, which seems like it must be THE_. But the frequency of 78 seems too high for Y and too low for N. The best guess, I suppose, is that it's Y and there are a lot of "coincidental" 78s. [EDITED to add: actually it was YOU and YOUR, duh.]

Answer 2

Looks like Gareth got it while I was working on my last edit! Glad my brain-stream helped.

Original Ramblings:

In a similar vein to the other answer, I don't actually have an answer.

I do, however, have some (potentially) interesting data I've gleaned from the data.

The distribution of digits remains the same (ignoring noise) for every digit as it is with every second digit, every third digit, etc... all the way up to every 26th digit. I looked at this for all initial offsets, meaning that there are seven ways to take every 7th digit, but all produce the same distribution.

As has been previously noted, the distribution for all digits (sorted by frequency) is:

6  598
5  515
7  487
3  325
9  271
4  216
8  178
1  127
2  125
0  65

Also 6 is the only digit that appears in a run of 4, 5 and 6 are the only that appear in a run of 3. Statistical analysis is not my strong suit, but it doesn't seem unreasonable that we could find such runs if the 2907 digits were completely random with the distribution we see. In fact, my gut tells me runs would be more common, especially of length 3. Related: 0 is the only digit that never occurs in a run, even of length 2.

Rabbit Hole

If we treat the digits 6 and 7 as prefixes, we get 26 distinct elements. By re-encoding these elements as single characters, we get the following block of text. Single digits (except 6 and 7) become a through h, elements of the form 6x (60 doesn't appear) become i through q, and 7x (76 doesn't appear) become r through z.

qdclewfqbhfieenfwebzehfpdyfwdlefqduywfknfwkcengeftkzqfqkwfcdnmfzqknfibgjfguyrehfdreyfbfgqelkgbcfrewwecfknftqkgqfqeftbwfiyetknmfbfobyzkgucbycaflbcdhdyduwfoydhugzffqkwfqebhftbwfwunjfuodnfqkwfiyebwzfbnhfqefcddjehfpydlflafodknzfdpfrketfckjefbfwzybnmefcbnjfikyhftkzqfhuccfmybafoculbmefbnhfbficbgjfzdojndzffwdftbzwdnfwbkhfqefwuhhencafadufhdfndzfoydodwefzdfknrewzfknfwduzqfbpykgbnfweguykzkewffkfmbrefbfwzbyzfdpfbwzdnkwqlenzffbgguwzdlehfbwfkftbwfzdfqdclewwfguykduwfpbguczkewfzqkwfwuhhenfknzyuwkdnfknzdflafldwzfknzklbzefzqdumqzwftbwfuzzeycafknevockgbiceffqdtfdnfebyzqfhdfadufjndtfzqbzffkfbwjehffadufweeflafhebyftbzwdnfkzfkwfndzfyebccafhkppkguczfzdfgdnwzyugzfbfweykewfdpfknpeyengewfebgqfheoenhenzfuodnfkzwfoyehegewwdyfbnhfebgqfwklocefknfkzwecpffkpfbpzeyfhdknmfwdfdnefwklocafjndgjwfduzfbccfzqefgenzybcfknpeyengewfbnhfoyewenzwfdnewfbuhkengeftkzqfzqefwzbyzknmodknzfbnhfzqefgdngcuwkdnfdneflbafoydhugefbfwzbyzcknmfzqdumqfodwwkicafbfleyezykgkduwfeppegzffndtfkzftbwfndzfyebccafhkppkguczfiafbnfknwoegzkdnfdpfzqefmyddrefiezteenfaduyfcepzfpdyepknmeyfbnhfzqulifzdfpeecfwuyefzqbzfadufhkhfndzfoydodwefzdfknrewzfaduyfwlbccfgbokzbcfknfzqefmdchfpkechwffkfweefndfgdnnegzkdnffreyafckjecafndzfiuzfkfgbnfsukgjcafwqdtfadufbfgcdwefgdnnegzkdnffqeyefbyefzqeflkwwknmfcknjwfdpfzqefreyafwklocefgqbknffxbffadufqbhfgqbcjfiezteenfaduyfcepzfpknmeyfbnhfzquliftqenfadufyezuynehfpydlfzqefgcuifcbwzfnkmqzffxcffadufouzfgqbcjfzqeyeftqenfadufocbafikcckbyhwfzdfwzebhafzqefgueffxdffadufnereyfocbafikcckbyhwfevgeozftkzqfzquywzdnffxeffadufzdchflefpduyfteejwfbmdfzqbzfzquywzdnfqbhfbnfdozkdnfdnfwdlefwduzqfbpykgbnfoydoeyzaftqkgqftduchfevokyefknfbfldnzqfbnhftqkgqfqefhewkyehfadufzdfwqbyeftkzqfqklffxfffaduyfgqegjfiddjfkwfcdgjehfknflafhybteyfbnhfadufqbrefndzfbwjehfpdyfzqefjeaffxmfadufhdfndzfoydodwefzdfknrewzfaduyfldneafknfzqkwflbnneyffqdtfbiwuyhcafwkloceffkfgykehffsukzefwdffwbkhfqefbfckzzcefnezzcehffereyafoydicelfiegdlewfreyafgqkchkwqftqenfdngefkzfkwfevocbknehfzdfaduffpydlfzqefbhrenzuyefdpfzqefhbngknmflenfiafwkyfbyzquyfgdnbnfhdacef

Might not be enlightening, but similar approaches may be fruitful. The python code used to generate the above is:

z = "<the string of numbers minus whitespace>"
prefixes = '67'
alpha = 'abcdef__gh?ijklmnopqrstuvw!xyz' # _'s in the positions of the prefixes (indexes 10 and 26 don't appear)
out = ''
currprefix = 0
for c in z:
    if currprefix == 0:
        currprefix = 10 * (prefixes.find(c)+1)
        if currprefix == 0:
            out += alpha[int(c)]
    else:
        out += alpha[currprefix + int(c)]
        currprefix = 0
print out

Rabbit Hole, part 2

Rosie hinted in chat that the decoded message consists of the alphabet, plus a space. (I am currently operating under the assumption that case is ignored.) This means we are looking for 27 symbols.

The current code I'm looking at is similar to the above, but generated to prefixes 5 and 6. Out of all possible prefix pairs, this is the only pair that has both exactly 27 symbols, as well as no runs of three. The code is here:

sdcwf nsbhobffypmb hfhogd gplwfohd d g nvypodcfygfpcv hsohv ncdyxphsvyobbguq d g afhl af gjqsfwvgbcpaf pmcodypcsvgsohfpcb nt gf cvyxjo b g hvg dcb gcaofbcdhd gd d nz gdh dg hnsv nsfbhpcb n pdyupdzdyohv nt gfb phjyhohfkddufhog gdwofao dvy hlepavf ckvufjpph gbyxfkbyuobv ghpcv hsr dccopgbao c dwbxfjyhjobcbguphdzuyd hn lpcb h lypjvhohfppdhhfycaid drdov ho  gdzd mphdody af phodypl d hsje gvgbypmg d gv hvf nodoj afjpphb g hlej phdyv ohwfy hnbgg d phdwfhj nvpcb n hdohdcwf png d gvd d nebg dc hvf n hsv n pdhhfyody h g d oddyody hdofaofd phody hvwb hfphsd dxs h n cb n d h hf gcaodyf fzcvgbtcfnsd clymb g hsrdid docyd cphsb hnvj ocfhnad dpmfofarfb gpcb h lyod hod nyd hpgfbccarveevg dc hphdqdy ph g dg hjpm gvf ndeodyef gfygf nfbgsrfzfyhfy hpdzdyod h nz gfhfgf pl gjyhmbgspodwzcfodyod h mcenveje hf grdvyxpllyfpodwzcaocydgu nd d hjccphsfqfy h gbcodyef gfygf nbyho  gf my h ndyf nb dhvfygfpcv hsphsfpphb g hvyxzdvy hjyhphsfqdygc d oddylyfofbao  gdh dgfjpphb g hcvyxphsd dxso d podtcajoff gf h gvgvd d nfeefg hnyd cod hpcb nyd hpgfbccarveevg dc hobajyody o fg hvdylephsfopgdd afobf h cffyid d gkfe hogd gfevyxf gjyhphs dwtphdogffcppd gfphsb hid drvhov ho  gdzd mphdody af phid d gpofbccqbzv hbcodyphsfolchogvfch nodpmfovqdyyfg hvdyn af gakvufcaov hob d hodqbypb dvgucapohd cid djqcd mqdyyfg hvdynsf gfj gfphsfofv podyxkvyu ndephsfpaf gapodwzcfqsbvyn  bnad dohbhqsbcuobf h cffyid d gkfe hogvyxf gjyhphs dwtpcsfyid dpgf h d gyfhog gdwphsfqc dtkb phoydxs hn  cnad do  d hqsbcuphsf gfpcsfyid do cbaobvccvb gh n hdpphfbhaphsfq dfn  dnad dow af go cbaobvccvb gh nf fgfz hpcv hsphs d g phdyn  fnad dphdchoffogd d gpcffu nbxdphsb hphs d g phdyohbhjylz hvdylyplwfpl d hsje gvgbyo  gdzf g hapcsvgspcd dchm fzv gfodyjofdy hsjyhpcsvgsohfrf od gfhid dphdpohb gfpcv hsohvwn  nid d gqsfguobdduod ncdgufhodyofar gb cf gjyhid dohb afov hj ocfhogd gphsfocfan  xid drdov ho  gdzd mphdody af phid d gofdyfaodyphsv nwbyyf gnsd cjt pd ghcapodwzcfnvq gvfhn b dv hfpln jvhohfjkv h hcfow h hcfhnf af gao  gdtcfwobfgdwf n af gaqsvchv ohpcsfylygfod hod nf fzcbvyfhphdid dne gdwphsfjh afy h d gflephsfrbygvyxoffyobapod gj g hs d gqdybyrdacf

The space character is chosen to be both the most prevalent and to result in the shortest length for the longest word (25 letters). If the triplet rule is ignored, the longest word can be trimmed down to 16 letters (prefixes 6/8).

Sadly, this doesn't appear at to be a simple cryptogram/vigenere cipher/etc.

Answer 3

Wrap-up: The Making Of Megan and the numeric cipher

This is not a solution to the puzzle, but provides notes from its poser. This type of answer has been approved by the community.

Caution: This post may contain spoilers.

Inspiration

Seeing that many cipher puzzles have been posted here and been well received, I decided to post one. By a cipher puzzle, I mean one which presents a ciphertext to be deciphered. However, I decided to do something different from most of the cipher puzzles I'd seen here, to avoid drawbacks that I saw in these puzzles.

In some of these puzzles, the authors had enciphered their plaintexts in trivial ways, so deciphering them is trivial and (I imagine) does not produce a lot of enjoyment. Some looked to me as if the authors had tried to overcome this drawback by enciphering the plaintext using two or more trivial methods in succession. If this is done, each encipherment (except possibly the last) must be so simple that the solver seeing its output will know that what he's done is right, even though what he sees is ciphertext which still needs to be deciphered. The trouble with this, in the examples I saw, was that each encipherment needed to use a method which produces an output far longer than its input, so the puzzle contains a huge ciphertext, but the information content revealed by solving is pitifully small.

Some use, for enciphering, methods that are perhaps harder to see, but they are apt to make the solver, in Lopsy's wonderful phrase, "throw spaghetti at the wall until it comes up English".

I didn't see the point -- for this puzzle of mine, anyway -- of enciphering more than once; wouldn't the two decipherment tasks seem like solving two puzzles rather than a single puzzle in two parts?

Decipherment is hard, by design; what should the cipher puzzle creator do, to make it feasible and fun for the solver to solve it? One way is to state the key explicitly, or set the solver a simple riddle which yields the key. But, as xnor says, "the decoding (with key) is just rote work after you solve the riddle". And again we have two separate things to solve, rather than a single puzzle.

I wanted my puzzle's main task to be
- identifying the method
- working the key out

After that, the actual deciphering is, inevitably, no more than rote work, but at least the solver will have had to do something harder than solving a riddle to reach that stage.

For the puzzle to be fair, the method must be one the solver might find by researching encipherment methods and sticking to those that would not be too hard to decipher. (Possibly modified in ways evident from the puzzle statement.)

Creative steps

So then, what encipherment method should I use? There are many encipherment methods which I haven't seen here, so I wanted to create a new data point of how hard a method is to decipher. The straddling cipher seemed ideal. It is like the familiar substitution cipher, but harder.

The solver should know to do a frequency analysis. With a straddling cipher, ciphertexts betray signs of the uneven frequency distribution of letters in English text, even if they are not as easy to interpret as those of a straight substitution cipher. This should help the solver see clues and avoid a pasta/wall impact.

I thought it would be nicer to encipher the spaces, not only to create an additional helpful sign, but also because, when at last the plaintext emerges, it is easier for the solver to read it with the spaces in.

What should the plaintext be? I'm not as creative as to tell a story of the kind in which someone would need to send someone else an enciphered message (e.g. espionage or forbidden love). And I didn't fancy trying. So, a text by someone else. I felt that, to be fair, it should be reasonably well known or easy to find.

Some explanations of the straddling cipher's key grid present the column labels in the order 0123456789. But, where I learnt of it, these labels may be in any order. I should make it feasible for the solver to get the key, so I must state a hint for this. What more natural hint than that the key involves the author. I must add that the rest of the key grid is filled out straightforwardly (just as we know how, for a Playfair grid).

So I needed an author whose name I could present in a form which has no repeated letters. Arthur Conan Doyle can be given as AC Doyle. At 8 characters, including the space (I am enciphering spaces!), this is exactly the right length for the "monofid" row (the row containing plaintext symbols, each of which is enciphered by just one digit). What's more, Doyle's dates are 1859-1930, making it natural to put 185930 into the digit row. To answer a point Gareth McCaughan raised, the 30 is not coincidence; if I hadn't added the thematic 30, I would have 023 instead of 302.

The obvious choice of plaintext is The Adventure of the Dancing Men: it's a well-known story where Sherlock Holmes cracks a cipher.

Why did you choose one format over another?

I dithered for ages over whether to just present the ciphertext, or to wrap it in a story -- the story would be unconvincing, but with no story, AUGH. I eventually decided to wrap it in a story involving my friend Megan who had appeared in two of my earlier puzzles. Megan is not technically-minded enough to construct cipher puzzles herself, so someone sent it to her. My story is thus a three-hander. Perhaps it would've been better as a two-hander, with me breaking the fourth wall to appeal for help from you, dear reader, rather than my friend appealing to me.

Having another narrator state some clues in an email had the bonus of enabling me to include even more hints for the solver:
- Unlike me, the e-mail writer puts two spaces between sentences. I thought this might make solvers test whether a digit could encipher space, by interpreting its double as a sentence break.
- In his e-mail's From-line I gave his name in the form two initials, space, surname, to hint that if he were to use "A. C. Doyle" as key, it would be as ac_doyle.

I broke the ciphertext up into 10-line paragraphs, and each line into 10-digit words, to make it look a little less daunting than a solid wall of digits. This was perhaps not the best choice: see Takeaway below.

Resources

Did you write a program to help you find the solution or to prove that no other solutions exist?

Indeed I wrote a program to both encipher and decipher. It also did a frequency analysis of the ciphertext: this brought to light an issue I discuss below.

Logistical steps

Did you do any research for this puzzle? How did you find or collect the resources needed for this puzzle?

Serendipity. Browsing blogs, I came across this story of a cipher which stumped the FBI. One phase of this cipher is a straddle-cipher, a method I had not come across before. It struck me that it would be suitable for this site. I mean just the straddle-cipher, without the additional phases which made a cipher which was hard enough to stump the FBI!

Did you need to modify aspects of the puzzle as you went along in order to make things work?

Not that it would have otherwise failed to work, but here's one thing I did to prevent it being harder than it need be.

With this method, the ciphertext consists of digits. A solver would naturally wonder if two digits encipher each plaintext symbol. To stop solvers getting that wrong idea, I adapted my plaintext so that the ciphertext's length was odd. This didn't quite have my desired effect: see Takeaway below.

My program's frequency analysis of the ciphertext showed that 5, which enciphers space, was close in frequency to 6 and 7, the prefix-digits. I had to check the ciphertext to ensure that it would be clear to solvers which of these digits did which job: that neither 6 nor 7 could encipher space, for, if it did, there'd be too great a variety of 1-letter words, or the occasional huge word. I am glad to see that Phlarx saw the possibility of 5 and 6 being prefixes, tried it, and (correctly) rejected it.

Evolution

How did this puzzle change from the time it was first posted (perhaps based on user feedback)? Did you add any hints?

I changed nothing, but, after 24 hours, added one hint. Even though nobody had solved my puzzle -- meaning that it had lasted about 16 times as long as any of my previous four -- I still felt that it was solvable, so I phrased my hint in the form of questions rather than facts.

Takeaway

What did I learn by posting this puzzle?

That such a cipher, with the hints as given above, can be cracked.

Is there anything you would do differently the next time?

I was in chat while some users tried to solve it. My presentation of the ciphertext as 10-digit "words" confused some and led to the unintended plausible possibility that each 10-digit word represents one letter. Perhaps it would've been better to not put spaces into the ciphertext. Perhaps it would have been still better to present it as one line, even though it would then appear with a scrollbar!

I had taken care to make the ciphertext's length odd. This turned out to be not good enough. Someone factorised the number of digits in the ciphertext, found it to be a multiple of 3, and wondered if three digits enciphered each plaintext symbol! Perhaps next time I had better make the ciphertext's length prime!

Were there things you thought would be well-received that weren't, or vice versa?

Glad I didn't get any adverse comments about the framing story.