Abstract
Purpose
To assess the reliability, validity, and use of the Family Star Plus, one of several Outcomes Stars increasingly used as part of outcomes-based accountability approaches in the delivery of family support services. The Family Star Plus measures progress towards effective parenting but a lack of evidence exists on its psychometric properties and suitability for use as an outcomes tool.
Method
Based on data from 1255 families receiving a pilot support service, Cronbach’s Alpha was used to assess the internal reliability of the 10-item scale, while a Principle Components Analysis (PCA) examined the number of constructs in the tool. Using matched data from an evaluation of 80 families, correlations between the Family Star Plus and psychometrically-validated tools were used to assess concurrent validity. Findings from a process evaluation explore practical issues around use of the tool.
Findings
Cronbach’s Alpha indicated sufficient internal reliability of the Family Star Plus, however the PCA raised questions concerning the internal validity the Star. Correlations between the Star and validated tools were not strong enough to support concurrent validity of the Star. Process Evaluation findings highlight inconsistencies in Family Star Plus data capture which may explain these differences.
Practical implications
Further work is required before the Family Star Plus can be considered for use as an outcome measure.
Originality
This is the first peer-reviewed analysis of the psychometric qualities of the Family Star Plus.
To assess the reliability, validity, and use of the Family Star Plus, one of several Outcomes Stars increasingly used as part of outcomes-based accountability approaches in the delivery of family support services. The Family Star Plus measures progress towards effective parenting but a lack of evidence exists on its psychometric properties and suitability for use as an outcomes tool.
Method
Based on data from 1255 families receiving a pilot support service, Cronbach’s Alpha was used to assess the internal reliability of the 10-item scale, while a Principle Components Analysis (PCA) examined the number of constructs in the tool. Using matched data from an evaluation of 80 families, correlations between the Family Star Plus and psychometrically-validated tools were used to assess concurrent validity. Findings from a process evaluation explore practical issues around use of the tool.
Findings
Cronbach’s Alpha indicated sufficient internal reliability of the Family Star Plus, however the PCA raised questions concerning the internal validity the Star. Correlations between the Star and validated tools were not strong enough to support concurrent validity of the Star. Process Evaluation findings highlight inconsistencies in Family Star Plus data capture which may explain these differences.
Practical implications
Further work is required before the Family Star Plus can be considered for use as an outcome measure.
Originality
This is the first peer-reviewed analysis of the psychometric qualities of the Family Star Plus.
| Original language | English |
|---|---|
| Pages (from-to) | 109-122 |
| Journal | Journal of Children's Services |
| Volume | 15 |
| Issue number | 3 |
| DOIs | |
| Publication status | Published - 10 Aug 2020 |