@StudioWildcard, please! I love the game, bought it and all expansions multiple copies even.
There's a Host Non-Dedicated Server option for hiding your server from others, but you aren't allowed to invite anyone in. This makes it exactly the same as Single-Player mode.
Please just look into it. It's for setting up your server to be private of others, but allows you to invite friends. (Xbox issue)