Beat the butter and sugar on medium speed until pale and fluffy, about 3 minutes. Add the egg, 2 tsp almond extract, and vanilla, and beat until combined.

Whisk the flour, baking powder, and salt together, then mix into the butter mixture in three additions until just combined. Shape the dough into a disk, wrap tightly, and chill for at least 30 minutes.

Preheat oven to 350°F (175°C). Roll the dough to ¼ inch thick on a floured surface, cut out shapes, and place on parchment-lined baking sheets. Bake for 10 to 12 minutes until edges are barely golden. Cool on the sheet for 3 minutes, then transfer to a wire rack to cool completely.

Stir the powdered sugar, ¼ tsp almond extract, and 2 tbsp milk together, adding more milk as needed to make a thick, drizzly icing. Decorate the cooled cookies, add sprinkles if using, and let the icing set for about 15 minutes before serving.