Part of why it doesn't serve as a deterrent is that we've sanitized it to the point of absurdity by hiding it from public viewing and going from hanging and electrocution to the comparatively "nicer" lethal injection, and then worried that even that might be cruel because the needle stick hurts or the condemned might have felt a bit uncomfortable before succumbing. On top of that, with an average time of 16 years between sentencing and actual execution, there's little connection between the crime and its consequences.
Capital punishment is supposed to be gruesome, painful and scary as hell to the criminal being executed so that it serves as a warning and deterrent to others. When you sanitize it, make it painless, hide it from public view, and then delay it so long that it has no effective connection to the crime that it's punishment for, how do you expect it to have any deterrent effect?
Back in the day, they hung murderers and cattle rustlers in the town square and they did it not long after conviction. It wasn't pretty to see Ol' Joe up there terrified and begging for forgiveness and promising not to ever do it again, and it wasn't pretty to watch his neck snap, his legs twitch, and his bowels and bladder empty, but it damn sure convinced a lot of people that they probably didn't want to commit murder or steal any cattle and end up like Joe.