Why does this happen?

It’s… man, the perspiration has dried on it. It’s the minerals that are deposited on the substance by perspiration. You perspire because your ears are warm and comfortable as you play. Remove the surrounds made with material (most can be replaced anyhow), and use some washing soap and a moist rag to clean them. After allowing them to dry, replace them.

I have no idea how to clean it, but that sure appears like a lot of oil accumulation or dead skin.