Preheat the oven to 350°F and grease a 9×13-inch baking dish. Whisk the flour, granulated sugar, and baking powder in a large bowl. In another bowl, combine the milk, eggs, vanilla, and slightly cooled melted butter. Stir the wet mixture into the dry ingredients just until the flour is incorporated. Spread the batter evenly in the prepared dish.
In a separate bowl, combine the softened butter, packed brown sugar, flour, and cinnamon into a soft spoonable mixture.
Drop the cinnamon mixture in small spoonfuls across the cake batter, including near the corners.
Use a butter knife to swirl the topping through the upper layer of batter in a few loose back-and-forth passes. Stop while the pale batter and brown cinnamon streaks are still distinct.
Bake at 350°F for 40 to 45 minutes, beginning to check at 40 minutes. Test a cake section near the center with a thin knife or toothpick, avoiding a large cinnamon pocket. It should come out clean or with a few moist cake crumbs, with no wet batter. Continue baking if necessary.
Let the cake cool in its dish for about 5 to 10 minutes. Meanwhile, stir the powdered sugar, ⅓ cup milk, and vanilla together until smooth and pourable.
Pour the glaze over the still-warm cake. Gently spread it toward the edges if needed, then let it settle for a few minutes.
Cut into about 12 generous squares, or smaller pieces as desired. Use a broad spatula to lift the soft cake slices.