The abbey suffered damage from a number of invasions by the Vikings and the Saracens, eventually being destroyed, once in 838 AD and again in 923 AD. BY 977 AD, the abbey had been restored.
Saint Isarn, a Catalan monk, began a new construction of the abbey in 1020 AD. Because the Abbey of St. Victor had a strong relationship with both Spain and Sardinia, (as well as owning property in Syria), it had developed into an important economic influence during the Middle Ages.
From the early to the mid-14th century, Pope Urban V enlarged the abbey, surrounding it with crenellated walls (making it look more like a Medieval castle than a monastery). By the early 16th century, however, its influence began to decline, partly due to the loss of its library’s rich ancient manuscripts. These had been dispersed by Abbott Giuliamo di Pierfronips de Medici, in order to please Catherine de Medici.
During the plague of 1720. The monks barricaded themselves behind the walls of the abbey, instead of giving assistance to the ill. As a result, Pope Benedict XIII, secularized the. Monastery, converting it into a collegiate church. The last Abbott died in 1787 and was not replaced before the French Revolution. By 1794, the abbey was stripped of its treasures and the building became a warehouse, prison, and barracks.
Today, all that remains of the original abbey is the church of St. Victor which was dedicated by Pope Benedict IX in 1040 and rebuilt in 1200.