HR, Fatigue, and Training Effect

This is a shout out to all you Exercise Phys experts.

Its a question I'm sure everyone has asked or thought about at some point training for an IM or half IM, but probably never looked any further. It crossed my mind this w/e as I struggled through a "speed set" in the pool as my third workout of the day, and at the end of a very heavy week of training and work. I was doing sets of 12x50 (25 all out, 25 easy) followed by a 200 all out. I was dead tired, but after starting I was determined I was going to finish the workout. During the workout I kept track of my HR and times. I was clearly at my LT based on my HR, but my times were VERY slow compared to what they should have been. As I struggled through, I kept wondering - am I actually getting the benefit of the workout I should be ? I was clearly working at my LT, but at the same time I was soo dead tired, I definately wasn't doing a real "speed set". So often when training for an IM we do workouts dead tired and just pushing through. So my question to the experts :

Do you still derive the benefit of pushing up your LT doing these workouts dead tired and slower than expected, but clearly at the proper HR zone ? Or are you doing more harm than good ? Or is it better to just call it quits, skip the w/o and have a nap and recover and do a better quality workout tomorrow ?

I would be interested in seeing if there is any scientific evidence - or everyones personal experience.