cute but seems to run big

I have 37 inch hips and a 29 inch waste and the small fit perfect but felt like it would most likely fall off in the surf or when diving. If they were a solid color I think I would have kept them and lived with it or sewn a drawstring in, but I got the black pair with white and black striped waistband and that part looked weird (you can''t keep them folded perfectly, so the stripes don''t line up and just look off). They were super flattering in the rear though, covered the upper part of my leg where leg meets bum (definitely a problem area for me).  Will keep an eye out to see if more colors become available and try again with an x-small.