Ito

Hello, my last name is ITO (伊藤). My family is from Japan and when my grandfather was about to die he give my the translation of our family name.
The /i/ (伊) has a connection to Ise Province [伊勢国 (Ise no kuni) or 勢州(Seishu)].
Today it includes the majority of the modern Mie Prefecture.
The translation of "wisteria" (藤) is totally true.
Finally the right translation of ITO (伊藤) is the Fujiwara clan in Ise.
(you wasn't far away at all ^^, just 140 miles)

There is also a theory that the /i/ (伊) has a connection to Izu (伊豆), an area on the southern coast of central Japan, and that the combination of this character with the "wisteria" part (藤) of Fujiwara indicates that they served the Fujiwara clan in Izu.