yeah, especially with the breath holding and swimming for long periods
actually yes, that's part of it. another part is not just swimming for long periods but holding a stable position in a formation, especially during times when the only thing you have to brace yourself against is water. it takes an incredible amount of energy to hold those positions while minimizing disturbance on the surface and that energy expended means a greater need for oxygen. you're not holding your breath for more than ten or fifteen seconds, they're doing it upside down and with a heartrate of a 100 meter sprinter.
I don't consider it a "sport" but it without a doubt requires athleticism