Duff was born in Houston, Texas as the second of two daughters of Susan Colleen, a homemaker, and Robert Erhard Duff, a partner in a chain of convenience stores. She has an elder sister, Haylie Duff, who is also an actress/singer. Duff's mother encouraged Hilary to take up acting classes alongside her elder sister, which resulted in both girls winning roles in various local theatre productions.
