Database Permission

Let say I added a document to my database and locked it as Private but there is one thing that I don’t understand all documents must be uploaded to my media manager so it does not make sense to make it Private in the database because still everyone can access to my media I mean it’s public.

How can I make my documents private in my media manager too. I just want to let some of my users be able to download it. If I know something wrong can you tell me where I’m wrong because I didn’t understand the point of the private database with documents from the media manager?