This would be the worst time for Justice Kennedy to step down