There is no 100% reliable way to counter this. You can stun the enemy but after stun passes it might choose to silence you again. There is also a late-game accessory that reduces silence duration by 50%.
At later stages of the game you have access to so many spells and your mana regen is so high you could just spam all spells and be done with any fight. Most fights with monsters with silence require a specific strategy like re-casting summon beforehand, using potions, using specific equipment, timing your spells correctly and so on.