Hi Fanta,
You can get him to stop without being rude by sarcastically joking with him in saying something like, “The next time you tell me to smile again, I am filing a complaint with HR.”
Give it a moment to see his reaction. It will most likely be stunned silence. Then just start laughing, wave your hand and say, “Just joking.”
Though you were joking with him, that will plant a seed of caution in his mind. You will most likely not hear him asking you that question from that point forward.