Anyway to have a members only chat with access to select people

So this is untested by me personally, but take a look at the Chatbox API

I have not tested a scenario such as yours, but there is a hide/show in the API so you could decide who sees the chat and sendMessage takes a channel ID so perhaps this can be used to route your messages? Take a look through the docs and see what you think.