I suspect that there’s someone here who can suggest an elegant solution. FWIW, I can suggest a crude solution that makes sense only if the number of categories is rather small.
That solution would involve creating one pre-defined gallery for each category and setting it as ‘collapsed’ (except, perhaps, for one gallery that would be the default, visible gallery when the page is loaded).
Then turn each category into a button that would run some Velo code to (1) expand the gallery for that category and (2) collapse any other expanded galleries.