On the history of Sanskrit

I am beginning to wonder what actual evidence there is that Sanskrit is as old as is usually claimed (5000 years, etc)

So far as I can see, we make this claim on the basis of imposing the view that Sanskrit "developed" from Vedic to Classical Sanskrit and so on

But I see no evidence for this. Once one begins to question the orthodox view that the Upanishads were compiled AFTER the Mantra Samhitas, and asks what is the evidence, one wonders why it could not have been the case that the Upanishads were compiled simultaneously with the Mantra Samhitas (if some groups of Aryans were, for example, more "mantric" and some more "upanishadic", as is the case even today?)

Specifically, on Panini: Why should it not have been the case that Panini was contemporary with the writers of the Vedas? Of course, we know nothing of when exactly Panini lived, and what we have are wild guesses ranging from the 8th century BC to the 4th century BC. But how do we know that Panini did not live in the 3rd century AD? After all the language of our earliest written records is Prakrit and, at least in north India, Prakrit was replaced by Sanskrit in our epigraphs only by the end of 3rd century AD. This seems to indicate NOT that Sanskrit is a very ancient language, rather that it developed FROM Prakrit, being therefore essentally a post-Buddhist language, which may have become established properly only with Shankaracharya's revolutionary activities in the 9th century AD, with what are now considered "orthodox" views being "read back" into history.

I will be very happy to have my questions answered one way or the other by evidence.