Whisk flour, yeast, sugar, pumpkin pie spice, and salt in a large bowl. Add warm milk, pumpkin puree, egg, and melted butter. Stir to a shaggy dough, then knead on a floured surface until smooth and elastic, 8 to 10 minutes.

Place dough in an oiled bowl, cover, and let rise until doubled, about 1 to 1 1/2 hours.

Punch down the dough and shape it into a smooth ball. Set it on a parchment-lined baking sheet, lay the oiled strings in a star pattern, bring them up, and tie loosely to create 8 sections.

Cover loosely and let rise until puffy, 30 to 45 minutes. Preheat oven to 350°F (175°C) near the end of the rise.

Bake for 25 to 30 minutes until deep golden and a thermometer reads 190°F (88°C). Cool 5 minutes, remove the strings, and insert the cinnamon stick stem.