I think it's important to remember that this is a magic trick, and most card magicians (like most folks in the general public) aren't entirely fantastic with maths. So... relying on a maths-based method is probably not the best plan. (particularly one that involves doing a modulus by 53 in your head while also giving an entertaining performance; I mean, it's flattering that you think I could do that, but it does seem just a wee bit optimistic.)
Personally, if I was going to perform this trick such that a spectator would describe it as was done in the OP, then I would use one of two methods:
(That's a lie; I'd use Method Two every time and sleep like a baby. But Method One would work and is provided here for completeness)
Method One (aka: the complicated way):
Before the trick began, I would remove the 9 of hearts, the 5 of diamonds, the 7 of spades, and the 4 of clubs. (Really, any cards will do, as long as they're unremarkable cards; don't use aces or face cards). Then, I'd re-seal the deck, complete with both jokers, the advertisement cards, etc. When presenting the deck, I'd make a big deal about removing one of the jokers and the advertisement cards as I introduced what was about to happen, and explaining that it was now a deck of 53 cards, including the single remaining joker.
Caveat: If you would feel bad about lying to the audience about there being 53 cards in the deck when there are actually only 49 cards (what sort of magician are you??), then you can include a second copy of the 9 of diamonds, the 5 of hearts, the 7 of clubs, and the 4 of spades (the alternate same-color-suits of the missing cards), to bring the deck's actual size back up to 53 cards. This trick will work the same, either way, but you're slightly cleaner at the end of the trick if you don't add these duplicate cards.
Anyhow. At the start of the trick, the two assistants would each have two cards; one would have the 9 of hearts and the 7 of spades, the other would have the 5 of diamonds and the 4 of clubs; the cards that we removed from the deck, earlier. These could be tucked away in a coat pocket or any other standard card-stashing positions. The assistants palm their cards while audience attention is on the spectator, who is selecting their card.
While being handed their half-decks after the spectator has made their selection, the assistants would then covertly add their palmed cards to the tops of their half-decks, and then do some false shuffles or whatever, and eventually would each "select" their two cards.
At the end, when the magician is given the shuffled set of five cards, the magician can easily tell that the spectator's card is the only one that isn't one of those four predetermined cards.
Further commentary on Method One:
In summary: the overall approach this method takes is to have four cards not in the deck at all, so that they can't be picked by the spectator. Then these previously-missing cards get added into the half-decks by the assistants, and then shuffled into the final set of five cards, so the deck invisibly becomes complete by the end of the trick.
As a result, you're entirely clean and everything can be examined at the end. Exception: if you added duplicate cards to the deck in order to keep the "53 cards in the deck" claim honest, then there are four extra cards in the deck at the end of the trick. In that case, maybe don't try to squeeze the cards back into the box while cleaning up; might be a bit of a giveaway if they don't fit back in. :)
It's also worth noting that as is common in magic acts involving assistants, the assistants here are the ones who have done all the difficult or skillful work; the magician does literally nothing, except to be engaging and to keep the audience's attention focused elsewhere while the assistants do their dirty deeds.
Method Two (aka: the easy way) (aka: untying the Gordian knot):
Just force the card.
Since you know what card they chose, everything else is just theater; you will trivially be able to identify that known card at the end of the trick.
Further commentary on Method Two:
Forcing a spectator to choose a specific card that you want them to choose, without them being aware that you've done it is an ancient trick. There are hundreds if not thousands of ways to do it. And in this case, it solves the entire trick; nothing else needs to be done.
Personally, I'm fond of using the Classic Force whenever practical. It requires a lot of practice to pull off with any sort of consistency, but if you can do it cleanly, it feels entirely fair and natural and nobody will ever suspect you've done anything untoward. Folks with less card manipulation skill (or audience-member-manipulation skill) may want to use the Hindu Force, or any other ordinary card force with which they are comfortable.
Under this approach, you're completely clean and everything can be examined both before and after (or even during!) the trick; there are no extra cards or missing cards or doctored cards involved. (unless you used them as part of your force, of course!)
It's worth noting that a lot of card tricks – maybe even the majority of them – follow this same "force the selection of a card, then provide an elaborate piece of theater to reveal the card" pattern. The theater is there to distract the audience; give them other things to be suspicious of, when the only real "trick" happened right at the start, with the selection of the already-known card.
