How referee should have responded...

Backfist to the future.

That's a bingo!

I was wondering how he could possibly get enough force behind it until I realized it's more a back-elbow than a backfist.

You got that right.