Peter is my brother's mn, and I've always loved it. I admit I have heard it used as slang for penis. It's rare, though, so rare it doesn't bother me. I asked my brother once if it had ever been an issue for him, and he said no. It's a great old name that deserves more attention.
When you think about it, many proper names have been used as slang for penis--Willie, Dick, Jack, and John Thomas, just to name a few--and most of them far more frequently than Peter.