Okay, perhaps not. I did not take into account that a day passed between the Potters were killed and Harry was placed on the Dursleys' doorstep (reference (http://www.hp-lexicon.org/timelines/timeline_potters2.html#So)). So perhaps Hagrid didn't tell Sirius where he was going... But he might still have done.