Looks like Temple got more votes in some polls than we did.
Think about it this way.
Say we beat Temple soundly, and then lose to UGA, Stanford, USC, and Miami. We end up 8-4 and unranked.
But if Temple beats everyone on their schedule except us, the best of which are Navy and Houston, they'll be 11-1, having rattled off 11 straight wins. They'll be ranked higher, even if no one thinks they could beat UGA, Stanford, USC, or Miami.
We may know for a fact that they couldn't complete with us and did not, but they'd definitely end up ranked higher.
Long story short, rankings are often dumb once you are outside of the Top 5-10 teams.