Throwing in multiple people into a scene where everybody contributes to the success of that scene means one thing for certain:
-things will go differently than you think they will.
This is bad news for controlling people (and chances are, if you're human, you're at least a little bit controlling). But, the good news is this--scenes are never limited to what you can come up with on your own.
A lot of times, in life, things seem to be static. The boss is always grumpy. The husband never helps out. There's never enough money. The kids are always ungrateful. Etc. Etc. Etc. Life never seems to change.
Improv reminds us that that attitude is wrong-headed. Things do change, sometimes drastically and sometimes subtly, but things are always changing--and, sometimes the littlest catalyst can make it all start happening.
A good rule of thumb for improv is this: expect the unexpected, and learn to like it. When a fellow improviser takes your "heart to heart" father-son talk into an outer space adventure you can either a) lament the loss of your Tony Award-winning scene idea, or b) get over it quickly, and start working on making the outer space thing work, or c) passively aggressively make everyone on your team feel like crap because you're a misunderstood genius.
I strongly recommend taking option B. Not only will it lighten up those around you, giving them reason to trust you, but it will make you a better/more flexible person. And, it will remind you to expect the unexpected. You may even learn to like it.