It's not improbable that extremely large and powerful animals might panic when confronted by certain types of much smaller animals. Take the classic example of the elephant thrown into a tizzy by mice.
For T-Rex, the panic-trigger took two forms: raccoons and cephalopods.
Creepiest neighbors ever! Really. Not kidding. One time he moved to avoid his raccoon and cephalopod neighbors, and they totally moved into the house next door!
And now at Hogwarts, he was supposed to be living next door to McGraw Tower, and
guess who lived there? Well, no raccoons yet, but. Cephalopods! Or something near enough as made no difference from T-Rex's vantage point!
And they
had talked to him. *Shudder*
T-Rex was kind of afraid to go home!
So he was standing around on the lawn, looking distraught, or as distraught as a tyrannosaurus can look. Wringing his tiny vestigial arm-hands.