If he asks me...how do I say no !!.....?

If he's loving and caring like you say and you've already told him you're not ready, then he won't ask. If he does, tell him you care about him very much but that you are not ready. That doesn't look stupid or idiotic. It looks like you're a strong person who only does what you feel comfortable doing.