Bishop Hay's episcopacy was dominated by the struggle to regain our freedoms and to shake off the sense that we were second-class citizens. He was the main negotiator with government officials at a time when it was still illegal for him to be in the country at all according to the anti-Catholic Penal Laws. Clearly attitudes were changing but it took time. English Catholics had acquired a Relief Act in 1779, lifting some of the Penal Laws, but this was too much for some and was a major cause of the Gordon Riots in London in 1780. It also contributed to Bishop Hay's house and new chapel in Edinburgh being set on fire. This put things back for Scottish Catholics who had to wait until 1793 for a Relief Act of their own. Bishop Hay retired in 1805 and died at Aquhorties in 1811. It wasn't until 1829 that the Catholic Emancipation Act was passed, but much of the credit must go to the efforts of Bishop Hay.