As in so many N64 racing games, Diddy's got a Turbo Start (like Wave Race, Mario Kart 64).
Just after the high-voiced announcer says "Get Ready" and the words start to fade, press the acclereration button (A). You'll get a mighty boost.
The boost is more powerful and lasts longer the later you start it. But be careful not to start it too late or it won't work at all.