If I understand correctly, you are drilling on the lathe and are not having any difficulties with the hole being centered. However, you do feel that its taking too long to drill the holes using a 3/8" or 14.5mm bit with the lathe running around 400 r/min.
You didn't mention two important considerations - first, what are you drilling? Many pens have separate cap and body sections that are made by cutting a typical 5-6" blank in half. Normally, the recommended approach is to cut the blank to length first, and then drill the two sections separately. Are you drilling the two sections separately, or are you drilling the blank before cutting it to the lengths required for the top and bottom halves of the pen? And of course there are pens that are made from only one blank, but that blank is longer than is the case with slimlines.
Second, you say that it takes 'forever', but how long is 'forever'?
When drilling pen blanks, you generally are drilling end grain, and that does take longer than drilling face grain. And the hole that you are drilling is pretty deep - easily 2" if you follow the recommended sequence (cut first and then drill), and easily twice that if you are doing it backwards. So it's going to take time. Presumably, you are following directions and periodically backing the bit out of the hole to clear the swarf and allow the bit and blank to cool a bit - and that's going to add to the time required.
By the way, the recommendation to use a lower speed when drilling is intended to limit heating. My experience is that you can sometimes get a cleaner hole if you run the lathe a bit faster - 500-700 r/min). But when doing that, its critical that you stop drilling to clear swarf more frequently. The heating actually comes about from swarf buildup in the flutes of the bit rubbing against the inside of the hole, so as you increase the rotational speed of the bit, you have to clear the swarf more often.
But it's also important to recognize that the speed at which the bit is advanced into the wood is a separate consideration from the rotational speed of the bit. If you try to advance the bit too rapidly, you will force the bit into the wood before the tip of the bit has a chance to cut a hole for the bit to move into. This can cause the bit to drift off center as it attempts to follow grain lines separating softer, summer-growth wood from harder, winter-growth wood. Obviously, this is much more of a problem with smaller bits, but it can also happen with hefty 3/8" bits.
Final thought - is your bit sharp? Drilling is faster if the bit is sharp, and trying to drill with a dull bit is a lot of work that creates the perception of taking forever.