If, and it's a genuine if,
Julius means "downy-bearded", then - male shaving habits being what they are - it should refer to the first whiskers a young gentleman sprouts. So there's your "youthful" idea.
I think we're actually up against the same mindset one sees with names like
Charlotte and
Andrea, where instead of thinking of them as the feminine forms of
Charles and
Andrew, meaning "male" or "manly", people prefer to gender-switch the definition and say that
Charlotte,
Andrea,
Caroline etc "really" mean "womanly" or "female".