Know that pain all too well. I made a lot of those learning how to manually line everything up without the luxury of a self-centering clamp. EVENTUALLY I got to a point where I could just eyeball it and get near-perfect results consistently.
I keep a stash of 2 1/2 inch blanks near my drill press (I think I cut them from some old scrap 2x4's). Whenever I have to make an adjustment to my drill press I drill one of these scraps to make sure things are still straight and true.
This is definitely the life of a segmenter..... Wait until you blow up a blank that took a few hours to make. I have blown up blanks that had 8 full cuts in them and then 8 glue ups. That hurts I can tell you that. It looks like you were way off center with the drill? I always mark center and then turn between centers then put the turned piece into the lathe chuck and drill. This way you are always on the center.