Refactor raw_hash_set deallocation to pass CommonFields instead of passing the results of a bunch of accessors of CommonFields.

Motivation: this makes it easier to refactor CommonFields to be smaller.
PiperOrigin-RevId: 551616928
Change-Id: I3710443fb156537d716944584bea02f945559e99