I think it's okay for both genders. I like it better on a boy right now, probably because the first one I ever met was a boy and it seems so trendy and boring on girls. I think I'd like Bayley a lot more for girls, it makes me think of a bay. I also like Bay as a full name for a girl and a nickname. Something really masculine as a middle name. Not Bailey Drew or Bailey Dale or something. Bailey William, Bailey Thomas, Bailey Joseph.
I don't think Azure or Clover have ever been common enough to be traditionally seen as anything, but they do sound a bit more feminine. I really dislike Brecken and Kenny so I definitely prefer Bailey over that.