It is complicated since the value presented in the collection is not part of the collection.
The real value in this field is the reference item ID.
Behind the scenes, the content manager queries the referenced collection and display its primary field value. This query doesn’t run for all the items in the collection which can be millions of items. It only runs for the current page items (if you scroll down it will load the next page and query the referenced collection and so on. So sorting the collection by values that are not in the current collection may be challenging.
But you can ask for this feature.
https://community.wix.com/velo/wishlist