The only thing I could find was (apart from like two forum posts) here
http://rep.up.krakow.pl/xmlui/bitstream/handle/11716/4887/AF080--11--Nazwiska-obce-i-hybrydalne--Kolber.pdf?sequence=1&isAllowed=y about the name Szeremet where it's said it comes from Eastern Slavic and ultimately from Ottoman šeremet meaning angry. However, it's only one source and it's just kind of mentioned in passing so it's hardly a definitive proof of where the name came from.