I wouldn't use it if you're uncomfortable with the HP associations because definitely that's what most people in English speaking countries will associate it with. (and how can you not like the books!...sorry, lol) But on the other hand, as others have said, it's use in the book does make it more acceptable, it's not a completely out there name any more and people know how to pronounce it properly.
Just a suggestion but have you considered
Hermia? it's a lovely variant, also Shakespearean and entirely free of HP associations.