Male and female unisexual flowers have repeatedly evolved from the ancestral bisexual flowers in different lineages of flowering plants. This sex specialization in different flowers often occurs within inflorescences. We hypothesize that inflorescence architecture may impose a constraint on resource availability for late flowers, potentially leading to different optima in floral sex allocation and unisexuality. Under this hypothesis we expect that inflorescence traits increasing the difference in resource availability between early and late flowers would be phylogenetically correlated with a higher level of sexual specialization. To test this hypothesis, we performed a comparative analysis of inflorescence traits (inflorescence size, number of flowers and flower density) in the sunflower family, which displays an extraordinary variation in floral sexual specialization at the inflorescence level, i.e. hermaphroditic, gynomonoecious and monoecious species. We found that species with a complete sex separation in unisexual flowers (monoecy) had significantly denser inflorescences. Furthermore, those species arranging their flowers in denser inflorescences also showed greater differences in the size of early and late fruits, a proxy of resource variation between flowers. Our findings support the idea that floral sexual specialization and consequently sexual segregation may be the consequence of different floral sex allocation optima driven by the sequential development of flowers that results in a persistent resource decline from earlier to later flowers.