Speech and/or language difficulties (SaLD) can potentially compromise a child’s health-related quality-of-life (HRQoL). Very few studies have examined associations between SaLD, other child and family factors and HRQoL and none have been undertaken in Australia. We explore these associations using data from a nationally representative Australian sample of 4–5 year old children, extracted from the Longitudinal Study of Australian Children (LSAC) (n = 4386). The Disability-Stress-Coping Model informed variable selection. Three domains of HRQoL were examined, and assessed on physical, emotional and social functioning subscales of the Pediatric Quality of Life Inventory (PedsQL). SaLD measures included parent concern about speech/language (Parents’ Evaluation of Developmental Status) and receptive vocabulary ability (adapted Peabody Picture Vocabulary Test-III). Multiple regression analyses revealed that various child and family factors representing all constructs from the Disability-Stress-Coping Model were significantly associated with HRQoL. Specifically, HRQoL was positively associated with parental warmth and child’s general health and negatively associated with parent speech/language concerns and maternal depression across all domains. Parents with concerns about their pre-school child’s speech and language rate the quality of their child’s health more poorly across physical, emotional and social domains. Associations between parent speech/language concerns and HRQoL were notable for being apparent in a (non-clinical) population sample and for persisting independent of factors such as maternal depression, parenting style and the child’s general health.