If you want to return to the original matrix, you need to keep it. There is no way to undo a random permutation, unless your data was ordered/structured in some way (was it?). If it was ordered, how so? The key word is random.
Just to be psychotic about it. randperm() is not random but pseudo-random. If you knew the state of the random number generator, it could be possible to go back to the original. I am guessing this is not what you had in mind.