If this liaison is not something that you actively seek, as opposed to something that your husband and this work colleague desire, then I would think hard before proceeding.

If you decide to go ahead with it, make sure it's about you, not just them.

__________________
Calm, quiet, smooth, devastating