You wouldn't necessarily find 'between-species' remnants, for lots of reasons. One, there may not *be* such things. Evolutionary change may happen quickly and dramatically. Mutation results in beneficial alteration, that increases survival trait. Say, binocular vision. If said mutation is inherited, and it significantly increases chance of survival, it will spread rapidly, and before long, there won't be any members left with monocular vision. And there might not be any intermediary steps, like 1.5-eyed creatures.

Secondly, when you're talking about the size of the planet, and millions of years of time, not a whole lot remains that we *can* see, just from the sheer wasting effects of weather and natural events over immense spans of time. We get to see the minute fraction of what survives in the geologic record from such distant times, surely not even one one hundredth of one percent of what was actually there.

And random chance doesn't necessarily 'require more time than a few billions of years'. Let's say you have 1000 6-sided dice and you roll them. What are the odds that they will all come up 6's? 6^1000 power, or 1.4166e+778 to 1 against it. Okay, pretty darned unlikely. However, it's just as likely to happen the very first roll of the dice as it is on the twelve trillionth time, or whatever. Obviously, the more rolls the greater the likelihood that it will happen (and reaching *certainty* that it will happen eventually, given enough time), but nothing at all says that something unlikely cannot happen quickly, even immediately. That's random chance for you -- you just never know when it will happen. That's what makes it random.

Also, you're thinking about this in a myopic kind of way. While it's not necessarily likely that a few billions of years would result in the random chance that begins life *on any particular planet*, there are trillions of stars, maybe more, with probably quadrillions or quintillions of planets and all of them got billions of years of rolling the dice too. So if it only happens on one planet out of a billion, because the odds are low, even so, there are a whole lotta planets. Could be we got one of the lucky rolls of the dice. I've seen no evidence to the contrary, anyway.