That's definitely not just your family. I've recognized that since I was very young. I was always the weird, quiet, analytical kid. My family gatherings were the same way, except mix in gossip too. I would listen to them and just shake my head. Even as a kid.
I hate gossip. I absolutely hate it. My family members would sit around and feed each other morsels of completely unfounded and invalidated horse crap, and they would each hang on every thought and word like it was gospel. It just infuriates me that people are that stupid. My immediate family realized it too though, so we never really kept in touch with our extended family while living overseas. That's just what happens when you grow up and stay in the same small town forever. You never evolve or achieve any wisdom or perspective on the world.
My mom used to tell me "Just think, that could be me. Thank God I met your father and we moved the hell out of there."
The funny part is that because my dad had the balls to go to college and make something of himself and earn his wealth he and my mom are now mostly ostracized and known only as "the millionaires". "The millionaires are too good to come here and spend time with us for the holidays." Shit like that.
Thank God we gave up on the family gathering garbage.