The question is a little unclear, but it sounds like you're asking for something like the example above, but whose solution is HA2QG.
One thing to be aware of right off the bat: The example uses only numeric digits, so each position has 10 possible values. Using an alphanumeric code increases that to 36 possibilities per position (26 letters + 10 numbers). That means you likely won't want to determine any of the characters by elimination, since that will require eliminating 35 possibilities in order to find the correct one.
I will use the following notation:
- Bold indicates the number of characters that are correct, and in the correct location.
- Regular text indicates the number of characters that are correct, but in the incorrect location.
So H8AGP (1, 2) means that one character (H) is exactly correct, and two characters (A, G) are correct, but in the wrong location.
Your request is a little ambiguous, because you ask for "the most complexity" and "the least amount of [clues]". These two are at odds.
If you are trying only to minimize the number of clues, then the obvious thing to do would be:
- HA2QG (5)
This doesn't make for much of a puzzle, since you are just providing the answer, but it does minimize the number of clues.
On the flip side, if you're aiming only for complexity, you can end up with a huge number of clues:
- BH7UK (1)
- BH7UL (1)
- BH7UM (1)
- BH7UN (1)
- BH7UO (1)
- BH7UP (1)
- ...continue, ad infinitum
The only information given by all 6 of the clues above is that the solution contains one of B, H, 7, U, and there is a single position eliminated for each of them.
You could use this method (or many, many others) to create hundreds of clues that, when combined would allow the solution to be discovered, but would take a lot of elimination and brainpower.
It sounds like what you're looking for is the right balance of conciseness and complexity, to make a relatively compact, but not-too-easy puzzle.
I would do something like the puzzle in your example, where you keep mixing up the characters in the clues, and each clue gives away only one or two bits of information.
With an alphanumeric puzzle, you could have a little fun as well by making your clues be words, or obvious plays on common words or phrases. For example:
- GHOST (2)
- QUART (2)
- HAREM (2)
- 2MUCH (2)
- SQUAD (2)
- DOING (1)
- UR2QT (2)
- GO2HL (1, 2)
Unfortunately, I don't think there's any algorithm that will give you the balance you seem to be seeking between complexity and conciseness. While there are certainly algorithms to create clues for a puzzle such as this, the clues will probably be uninspired and either trivial to solve or unnecessarily complex. I think your best bet is just to create your own clues. You might want to look at Clever ways to solve Mastermind? for some ideas as well.