Skyler's got it pegged. People have open relationships, and do just fine....but it takes the right sort of person. If you're not sure yet, you're not ready.

Better to call the thing off now and deal with the embarrassment of backing out then to discover afterwards that you can't handle what happened.

If you're looking for a middle of the road option, you could try some mutual voyerism. No cross couple touching....but everyone gets to see and possibly verbally interact with each other?

In any case, before the clothes come off...I'd definately have some serious talk time with everyone involved and figure out what the house rules are gonna be.

Good luck!