A color-coded sentence

Question

The puzzle is self-contained in the image below. The answer is a three-word sentence (7,4,6).

---

Here is a transcript of the image: for each cell the UPPERCASE letter represents the background color, the lowercase letter is the content of the cell. UPPERCASE letters meaning: B(blue) G(green) K(black) O(orange) R(red) T(turquoise) Y(yellow) V(violet) W(white)

code: T R K B V

Yp Yj Gx Rl Rj Tw Wx Vg Tc Re Kl Vg Wy Vc Bx Oa
Rb Yx Gs Kx Ww Rr Oy Gz Bt Te Vx Vu Kz Vi Ba Rx
Tl Tl Os Kt Ow Gl Yx Gt Te Bx Br Ga Wc Ve Rx Tx
Vn Bo Yl Yf Ro Ym Wd Rs Kx Bd Gm Op Yx Yu Bm Vo
Gx Oa Vi Tn Rx Ke Tw We Re Wz Vt Ow Vw Bx Tb Gi
Gm Tf Vn To Rx Vw Ga Yx Vs Ym Wc Kx Rr Gb Tx Bw
Tu Kv Rk Th Tb Rt Ks Vx Tr Wz Gm By Bw Kt Wo Vm
Yx Ym Rx Bo Ke Oo Bv Gs Yx Bx Oc Kt Ta Gw Vx Ov
Bx Oy Te We Kn Wu Yd Vi Tl Ge Bo Vm Gn Tm Rx On
Ob Tz Rn Bk Gf Va Bn Yx Wn Tt Te Bj Yd Kx Ri Br
By Gv Rs Be Gg Wk Ta We Gy Wq Be Vx Ye Gi Va Yd
Ts Bn Vp Go Yw Rx Rx Yg Tr Yj Tz Wq Wy Kx Or Be
Yt Rf Vt Th Gt Vi Od Re Tn Bx Kc Wo Vh Gr Bo Vd
Km Re Rd Os Ri Re Te Tf Bj Wb Rx Rh Vt Yx Wq Bi
Yu Vj Rn Ge Bc Ky Vo Rn Yt Kl Vl Rm Vj Ke Vx Ra
Ry Vi Gr Bh Od Oc Ge Vy Yx Vo Of Gc On Bo Wx Kt
Yh Vd Tl Kp Va Re Ga Ov Vz Tq Oc Tp Rw Bi Oj Gr
Rs Tu Rf Yd Th Kd Bp Yu Tm Os Yh Wm Oo Tr Vu Rp
Ou Vq Kv Gv Tx Ok Tx Wn Od Wa Tw Kg Ot Yx Vo Vj
Bk Vi Wg Rv Wy Wd Ok Rr Rj Rs Vp Vi Kr Bz Ov Kx
Wc Rm Ow Ry Vc Vf Bx Vc Wu Ge Oz Gp Ks Rf Gu Rf
Gq Rr Gs Kf Gy Gh Kr Kk Wg Va Bb Yo Ya Kf Ok Ol
Bi Km Rb Tx Yy Tt Tg Ot Vy Vs Wt Ve Bo Rk Yw Re
Vv Yq Rr Vq Wv Rf Gr Or Rg Gb Oq Kp Kf Op Gz Yk

---

Progress confirmation

I won't give hints for this puzzle but I will confirm your progress here until you find the solution.

1.

thanks to @LukasRotter in the comments: the distribution of the letters in the image is almost the expected one from the English language, except that there are way more xs than the expected number. In fact, the letter x has a particular meaning for this puzzle.

Answer 1

The encoded sentence is:

WELCOME BACK MELFNT! (where Melfnt is the op's user name)

Explanation:
The 'code' consists of 5 colors, T R K B V (using op's abbreviations).

If we start with the first T square, it is a "W". Then going left to right, top to bottom to the next R square, it is an "E". The next K square is a "L". These are noted in the image with yellow arrows pointing down and to the left.

Continuing, we get a message that is mostly intelligible, but contains a lot of X's. If we remove all the X's, we are left with: WELL DONE NOW REVERSE THE CODE YHL…. (gibberish)
Reversing the code gives the sequence V B K R T, shown in top right. The gray arrows pointing down and to the left spell out in the same manner:
GREAT NOW TRY TANZANIAN FHIE… (gibberish)

Let's look at the Tanzanian flag:

The blue could be turquoise or navy, some trial and error leads to the correct sequence: G Y K Y B
I didn't annotate this one but it spells out:
ALMOST DONE NOW TRY THE THIRD DUKE OF YORK
Richard of York is the third duke of York. He is associated with the mnemonic "Richard of York Gave Battle in Vain", or ROYGBIV, the rainbow. I'd never heard of this (not taught in USA) so it took some thinking to find.
Some trial and error gives the next sequence: R O Y G B V, as it's not quite ROYGBIV. This is marked with red arrows pointing up and to the right: LAST STEP NOW INVERT THE SECOND CODE VDVKI… (gibberish)
More! Does it ever end? Here we must sort out what is the 'second code' and what is meant by 'invert' (there are a few different definitions of inverting colors, depending on your color profile).
A lot of trial and error leads to the second code being the 2nd code where we reversed the original code and 'Invert' meaning 'Complimentary color in RGB mode'. Giving the new code:
G Y W T R, which spells out (annotated with yellow arrows pointing up and to the left):
WELCOME BACK MELFNT VEQHERQLE… (gibberish)

Answer 2

Wrap-up: The Making Of A color-coded sentence

This is not a solution to the puzzle, but provides notes from its poser. This type of answer has been approved by the community.

Caution: This post definitely contains spoilers.

---

I did not visit PSE the last year, so I wanted to create a good puzzle for the time I would be active again. I chose to make a puzzle with a single mechanism and a lot of passages, the main difficulty is to find the purpose of the "code", but once you figure it out it's just a matter of finding the next "codes" and then apply them to check if they are correct. This is rather simple because you already know the mechanisms, so I hope solving this puzzle gave you a lot of satisfaction.

Here are the step I used to create the puzzle:

First of all, I prepared in advance the sentences and the associated codes. For example suppose the sentences and codes were:
- sentence 1: "hello world" - code 1: GKBRG
- sentence 2: "the pen is on the table " - code 2: WRTK
- sentence 3: "seven little dwarves" - code 3: WKBRB
I had to create an image where the first G cell must contain an h, the following K cell must contain a e and so on, so that the solvers can read sentence 1. Also, the first W cell must contain a t (because of sentence 2) but it also must contain an s (because of sentence 3). This is clearly not possible: I called this situation a "conflict". To solve conflicts I introduced the "ignore all the x" rule: in this way the first W cell is an x (so that the solvers ignore it when reading both sentence 2 and sentence 3), then next R cell after the first Wx is a t (sentence 2) and the next K cell after the first Wx is a s (sentence 3).
I also made sure the sentences I prepared did not contain any x to avoid confusion.

Here is an algorithm to construct the image...

...given in input the sentences and the associated codes. Continuing the example above, only consider the first pair color/character of each sentence. For example, for sentence 1, the first G cell must contain an h. I say that sentence 1 "requires" Gh. Here are all the cells required by all the sentences: Gh, Wt, Ws.

pick a random color (call it RC) that will be the color of the first cell of the image. Four cases can occur depending on the requested colors:

RC is not requested: in this case just pick a random letter (call it rl), the content of the cell will be rl.

RC is requested by exactly one sentence (for example if RC is G in the setting above): in this case the letter of the cell will be the only one requested. After writing the cell, "advance" both the letter and the color of the selected sentence so that the next time it is selected the cell will have the right content. Continuing the example, if RC is G, the next cell in the image will be Gh, then "advance" the first sentence so that it requires Ke for the next cell.

RC is requested by more than one sentence, and the requested letters by those sentences are not the same (there is a conflict). For example, if RC is W in the setting above. In this case the letter of the cell will be x. "advance" all the colors in the codes requesting RC but keep the same letter: the x will be ignored by the solvers, while the requested letter for each sentence will be written in the image in one of the following cells. Continuing the example, if RC is W, advance both sentence 2 so that now it requires Rt and sentence 3 so that now it require Ks. Note that this may not entirely solve the problem of having an x, for example the new requested cells can conflict with other sentences. If many colors in the codes match there will be a lot of xs in the image, but I'm positive that in the end all the conflicts will be solved.

(this happens rarely) RC is requested by more than one sentence, and the corresponding letters are the same: the letter in the cell will be the the requested letter, "advance" all the sentences involved. Actually, point 2. can be considered a particular case of this one.

Repeat from point 0. (choose another random color RC...) until all the characters of all the sentences are terminated. One can run this algorithm manually but it takes a lot of time, so I wrote this python script that does all the work, including generate a png of the image and its transcription (Look ma, a lot of functional programming!):

import itertools
from collections import namedtuple
import random
import string
import math

from matplotlib.colors import ListedColormap
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np

Sentence = namedtuple("Sentence", ["text", "code"])
colors = "ROYGBVKTW"
first_code = "TRKBV"

chipher_cmap = ListedColormap([[1,0,0],[1,.65,0],[1,1,0],[0,1,0],[0,0,1],[1,0,1],[0,0,0],[0,1,1],[1,1,1]])

def invert_code (code: str):
    inverse_dict = {'R':'T', 'Y':'B', 'G':'V', 'B':'Y', 'V':'G', 'K':'W', 'T': 'R', 'W':'T'}
    return map (lambda c: inverse_dict[c], code)


sentences = [
    Sentence ("Well done, now reverse the code!", first_code),
    Sentence ("Great! Now try Tanzania", list(reversed(first_code))),
    Sentence ("Almost done, Now try the third duke of york", "GYKYB"),
    Sentence ("Last step, now invert the second code", "ROYGBV"),
    Sentence ("Welcome back, melfnt", list(invert_code(reversed(first_code)))),
]

def generate (seed=None):
    random.seed (seed)
    # print ("DEBUG: sentences", sentences)
    requests = list(map (lambda s: Sentence ( 
            filter (lambda c: c in string.ascii_lowercase, map (lambda c: c.lower(), s.text)),
            itertools.cycle (s.code) ),
        sentences))
    next_chars = list(map (lambda r: next(r.text, ''), requests))
    next_colors = list(map (lambda r: next(r.code), requests))
    char_count = 0
    collision_count = 0
    output=[]
    while not all (map (lambda c: c=='', next_chars)):
        # print ("DEBUG: sentences\n  {}".format("\n  ".join(map(lambda s: s.text, sentences))))
        # print ("DEBUG: iteration {}".format(char_count))
        # print ("DEBUG: next chars            ", next_chars)
        # print ("DEBUG: next requested colors ", next_colors)
        # requested colors have bigger probability to be extracted
        weights = [1+next_colors.count(c) for c in colors]
        color = random.choices (colors, weights)[0]
        # print ("DEBUG: random picked color ", color)
        indexes = []
        for i,(r_col,r_char) in enumerate (zip(next_colors, next_chars)):
            # find and count the sentences whose next color is the same as the random-picked color, ignoring the exhausted sentences
            if r_col == color and r_char != '':
                indexes.append(i)
        # print ("DEBUG: index of sentences requesting color ", indexes)
        if len(indexes) == 0:
            # print ("DEBUG: no sentence is requesting that color, emitting a random character")
            oc = random.choice(string.ascii_lowercase)
            # print ("{}-{}".format(color, oc))
            output.append ((color, oc))
        elif len(indexes) == 1:
            # print ("DEBUG: exactly one sentence is requesting that color, emitting the corresponding character")
            i = indexes[0]
            oc = next_chars[i]
            # print ("{}-{}".format(color, oc))
            output.append ((color, oc))
            # advance character and color iterators for the emitted sentence
            next_chars[i] = next(requests[i].text, '')
            next_colors[i] = next(requests[i].code)
        elif all (map (lambda i: next_chars[i]==next_chars[indexes[0]], indexes)):
            # print ("DEBUG: there are many sentences requesting that color, but the corresponding characters are the same")
            i = indexes[0]
            oc = next_chars[i]
            # print ("{}-{}".format(color, oc))
            output.append ((color, oc))
            # advance characters and colors iterators for all the emitted sentences
            for j in indexes:
                next_chars[j] = next(requests[j].text, '')
                next_colors[j] = next(requests[j].code)
        else:
            # print ("DEBUG: there are many sentences requesting that color and the corresponding characters are different (emitting an x)")
            oc = 'x'
            # print ("{}-{}".format(color, oc))
            output.append ((color, oc))
            collision_count += 1
            # only advance colors iterators for all the failed emissions
            for j in indexes:
                next_colors[j] = next(requests[j].code)
        char_count += 1
    print ("Done (seed={}).\nStatistics:".format(seed))
    print ("character count (sequence length): {}".format(char_count))    
    print ("x count (number of collisions): {}".format(collision_count))    
    print ("x percentage (the lower the better): {}".format(collision_count/char_count))
    return output, collision_count/char_count

def list_to_image (img_list):
    n = len(img_list)
    # print ("DEBUG: list size: ", n)
    w = math.floor(math.sqrt(n))
    h = math.floor(n/w)
    while w*h != n:
        w = w-1
        h = math.floor(n/w)
    print ("best layout: width {} height {}".format(w,h))
    
    a = np.array(list(map (lambda c: colors.index(c[0]), img_list)))
    data = np.reshape (a, (h, w))
    a = np.array(list(map (lambda c: c[1], img_list) ))
    annot = np.reshape (a, (h, w))

    print("\n".join(map(lambda row: " ".join(map(lambda c:c[0]+c[1], row)), np.reshape(img_list, (h,w,2)))))

    plt.figure(figsize = (w,h))
    sns.heatmap (data, cmap=chipher_cmap, vmin=0, vmax=len(colors), annot=annot, cbar=False, fmt='s', xticklabels=False, yticklabels=False, square=True)

if __name__ == "__main__":

    for seed in range(42, 50):
        out, _ = generate(seed)
        
        list_to_image(out)
        plt.show()

In addition, the script:

• tweaks the probabilities before choosing RC such that requested colors are more likely to appear, thus reducing the total number of cells in the final image.
• seeks for an "almost square" layout for the image (for example, if the output is 384 characters it creates a 26x16 grid rather than a 192x2 grid), because it is prettier to show.
• repeats the process ten times with different random numbers (i.e. using 10 different seeds). Is up to the human to choose which image is better.
• prints the density of x for each one of the trials.

Takeaways

Creating this puzzle was very funny as it gave me the opportunity to dust off my functional programmer skills -- just joking. At first I did not thought it would be that difficult to solve (it took more than one week which is long for this community). I also thought that the passages between the codes were trivial, but in the end I appreciate that they spiced up the process a little bit for the solvers.