There must’ve been something you didn’t like about your ex if you didn’t get back with him then. Often, people remember just the good things that happened in past relationships and forget about the bad. As res.5 said, maybe what you should do is call him or see him to remember what you didn’t like about him or....just to けじめをつける. If you’re thinking about another man in a marriage, it wouldn’t be fair to your husband, would it?
I think you are just a greedy woman. You want a secure marriage life and a love affair. You must have been grown up in the poor family or didn’t get enough attention from your parents when you were kid.