Don't forget that 'berg/burg/borg/bourg' morphed from meaning 'mountain' to 'hill fort' to 'fortress' then 'city' and 'town' (see link 1). It sometimes is used in districts of large European cities, especially the old royal or fortified quarters (Amalienborg in Copenhagen).
The name in America is often, but not intrinsically, Jewish in origin. These Ashkenaz names sometimes appear to be constructed without logic. Maybe they made sense at the time (see link 2).
http://dictionary.reference.com/search?q=borough http://www.ancestry.com/search/SurnamePage.aspx?html=b&ln=Katzenberg&sourcecode=13304