Grease a 9 × 13-inch baking dish. In a large bowl, whisk together the milk, cream, eggs, molasses, maple syrup, vanilla, cloves, ginger, and cinnamon for the casserole.
Arrange the brioche cubes evenly in the prepared dish. Pour the custard over the bread and toss gently to coat.
Cover tightly with plastic wrap and refrigerate for at least 5 hours or overnight.
When ready to bake, preheat the oven to 350°F. In a bowl, use your fingertips to mix the brown sugar, flour, butter, cinnamon, and ginger for the topping until moist crumbs form.
Remove the plastic wrap, sprinkle the crumbs evenly over the casserole, and cover the dish with foil.
Bake for 30 minutes. Remove the foil and bake for another 30 minutes, until the edges are toasted and the center is set with no loose custard. Serve warm with maple syrup or whipped cream, if desired.