In the normal world, of course, Hagrid would at the least be put on probation for what happens to Draco in his first class. It always amazes me when people even suggest Draco lies about what happens in the class to get Hagrid in trouble, as if the truth somehow makes Hagrid look good. (Or as if Draco's not hearing one throwaway line is a shockingly bad thing to do, even though Harry doesn't pay attention plenty of times in class, as do all kids.)

I think the text makes this very clear. Hagrid does not just have a problem with Malfoy; Malfoy is just the kid that speaks up. We're often told the other students agree with him, that they don't like Hagrid as a teacher--it's hard to believe Hagrid doesn't have plenty of more Malfoys in all his other classes. In sixth year it definitely seems implied that the whole school gets out of Hagrid's classes as soon as they can. Even the Trio prefers Grubbly-Plank for the very reasons they deny when not wanting to agree with Malfoy. Snape, by contrast, is mean but the students seem to respond to him as a teacher. His treatment of Harry is unbelievably out of line and inappropriate, but apart from him he just seems to be a very hard teacher who is mean to you if you mess up.