Diversity has to do with a lot more than just biology, religion, place of birth, and sexual preference/identity. And purposely built into the constitution is the ability to adjust, in order to adjust with the population and 'the times' (so to speak).
The founders were pretty smart people.