“By that time, Sauron had mastered all Eriador, save only besieged Imladris” Unfinished Tales: p239
After the death of Celebrimbor and the ruin of Eregion, Elrond ‘had gathered such few of the Elves of Eregion as had escaped, but he had no force to withstand the onset’ of Sauron. Elrond escaped when Dwarves out of Khazad-dûm and Elves from Lorien attacked Sauron’s forces in the rear. Elrond was forced North and took refuge at Imladris, where Sauron left a strong force to besiege him and prevent him coming to the aid of Gil-galad in Lindon.
So here is Elrond in that first, desperate siege, before Rivendell had bridges and buildings and defences, but was instead an emergency camp and refuge for Elrond’s rescue force and the refugees of Eregion. It was three years before the siege was relieved by Ciryatur of Numenor.