Mark Pattison (1813–1884), Rector (1861–1884) 너무 이른 다음 것