I do like Rufus. It's a warm name and it makes me think of Rufus Wainwright, whom I love. Its biggest negative for me is that it also makes me think of roofie, slang for the date rape drug.
It's funny. I've always considered Rufus to be one of the most obviously "red" of the red meaning names. To me, the connection between Rufus and Scarlett is like a big, red flag--not subtle at all.