ASSESSMENT OF A HEALTH PROGRAM DATABASE: THE BRITISH COLUMBIA PREGNANCY OUTREACH PROGRAM by Karen Davison B.A.Sc., University of Guelph, 1990 R.D.N., Vancouver General Hospital, 1991 C.D.E., Canadian Diabetes Association, 1995 THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE m COMMUNITY HEALTH © Karen Davison, 1998 The University ofNorthem British Columbia July 1998 All rights reserved. This work may not be reproduced in whole or part by photocopy or other means, without the permission of the author. r---------------~ -r ... - ......... . -:...,. ••.• · -~" iii ABSTRACT The mandate of the British Columbia Pregnancy Outreach Program (POP) is to provide education and support to high risk pregnant women. The purpose of the POP is to reduce the incidence of adverse birth outcomes. The initial intent of this study was to use the 1994/95 POP database as a basis for an epidemiological investigation of the relationship between the identification of risk factors and actual birth outcomes. In the course of the initial analyses, it was discovered, however, that significant data quality issues precluded any undertaking of the proposed research. The focus of the research was redirected to a methodical assessment of the database, using a conceptual framework outlining criteria for assessing such databases. In order to establish the database's utility within these areas, a review of each variable of the database was made in terms of record completion rates, verification of internal consistency, and assessment of coding errors. The findings of this investigation revealed that completion of data for each variable was consistently low. In addition, measurements of internal consistency indicated inaccuracies. In the process of the analysis of the database it was clear that definitions of variables required clarification and methods to reduce coding errors within the database needed to be introduced. Currently, stakeholders of the POP are reviewing the evaluation of the Program and its corresponding database. This thesis has documented fully the problems encountered in the database with respect to every variable that is entered. Ideally, the next phase of the development of this database would address the data quality issues identified in this investigation in order that the resulting information could be analyzed with confidence. lV TABLE OF CONTENTS Page APPROVAL 11 ABSTRACT lll TABLE OF CONTENTS IV LIST OF FIGURES Vlll LIST OF TABLES lX ACKNOWLEDGEMENTS Xl DEDICATION xu Chapter 1 -INTRODUCTION The Pregnancy Outreach Program OfBC POP Goals And Objectives Improve Nutrition Decrease Smoking Decrease Alcohol Use and Drug Misuse Raise Self-esteem Promote Dental Health Encourage Physical Activity Encourage Early Physician Care Promote SociaVCommunity Support POP Service Delivery Model POP Administration Sponsoring Agency Local Advisory Committee Local Health Department And Alcohol And Drug Programs Provincial Advisory Committee The Concept Of Comprehensive Prenatal Care Thesis Focus 1 5 5 5 5 6 6 6 6 6 7 9 10 10 10 10 11 11 Chapter 2 - THE PROVINCIAL EVALUATION OF THE PREGNANCY OUTREACH PROGRAM Organizational Principles Of The POP Database Management Of Client Data For The Provincial Evaluation Epi Info Version 5 Software Program Status Reports 15 16 19 20 v Page Summary 22 Chapter 3 - LITERATURE REVIEW - RISK FACTORS AND RISK SCREENING Risk Scoring Systems Risk Factors Pregnancy Outcome Summary 23 26 27 29 Chapter 4 - LITERATURE REVIEW - PROGRAM EVALUATION Evaluations Of Outreach Programs For High Risk Pregnant Women Assessment OfHealth Program Databases The Utility Of The Database For Program Evaluation Utility For Decision-making And Policy development Selection Of Information System Utility For Health Research A Conceptual Framework 30 32 33 37 37 39 41 Chapter 5 - METHODOLOGY Data Selection Formatting OfData Data Analysis Individual Prenatal Risk Identification Tool (IPRIT) Client Tracking Forms (CTF) T-ACE Questionnaire Tabulations OfEach Variable Frequency Distributions Record Completion Rates Internal Consistency Information Selection 43 43 44 45 45 45 45 46 46 46 47 Chapter 6 - RESULTS - INDIVIDUAL PRENATAL RISK IDENTIFICATION TOOL Determining The Number Of Completed Files The Individual Prenatal Risk Identification Tool - Description IPRIT - Section One: Physical Factors PFl -Previous Pregnancy Loss PF2 - Illness/Condition With Impact On Pregnancy PF3 - Pre-pregnancy Weight PF 4 - Rate Of Weight Gain 48 48 49 51 52 52 53 vi Page PF5 -Inadequate Nutrition PF6 - Previous Child With Anomaly Or Disorder PF7 - Previous High Risk Infant PF8 - Multiple Pregnancy PF9- Birth Interval PF10- Grand Multipara PF11 -Established Genetic Risk PF12- Age 17 And Under/Age 36 And Over At Time OfDelivery IPRIT - Section Two : Substance Abuse Risk Factors SAP 1 - Smoking SAP2- Alcohol Use SAP3- Inappropriate Use Of Over The Counter And Treatment Drugs SAP 4 - Other Drug Use IPRIT- Section Three: Socio-Economic Risk Factors SEF1 -Single Parenthood SEF2 - Delayed Access To Prenatal Care SEF3 -Refusal/Resistance To Appropriate Services SEF4- Isolation- Ethnic, Language And/Or Social SEF5 - Limited Learning Ability/Illiterate SEF6 - Marital Problems/Unstable Relationship/Family Violence SEF7 - Inadequate Housing SEF8 - Financial Problems IPRIT - Section Four: Emotional Risk Factors EF1 -Family History Of Abuse/Neglect EF2 - Mental Health Problems EF3 - Low Self-esteem EF4- Inability To Cope/Anxiety Regarding Pregnancy And Baby EF5 - Unrealistic Expectations EF6 - Unwanted Pregnancy Summary 54 54 55 55 55 55 56 56 57 58 59 60 60 61 62 63 63 64 64 64 64 64 65 66 66 66 67 67 67 67 Chapter Seven - RESULTS - CLIENT TRACKING FORMS Client Tracking Form- Section One: Program Information Client Tracking Form - Section Two: Referral Data Client Tracking Form - Section Three: Intake Data Client Tracking Form - Section Four: Client Characteristics Client Tracking Form - Section Five: Past Pregnancy Data Client Tracking Form - Section Six: Client Monitoring Client Tracking Form - Section Seven: Project Contact Client Tracking Form - Section Eight: Referrals Client Tracking Form - Section Nine: Program Outcome Client Tracking Form - Section Ten: Alcohol Data Client Tracking Form - Section Eleven: Smoking Data 69 71 71 72 76 77 85 87 88 92 96 Vll Page Summary 99 Chapter Eight -CONCLUSIONS AND RECOMMENDATIONS Individual Prenatal Risk Identification Tool Coverage Internal Consistency Specificity Of Risk Factor Definitions Client Tracking Forms Coverage Internal Consistency Specificity Of Definitions Provide For The CTF Evaluating The Database For Evaluation And Research Explaining The Problems And Issues Inherent In The Database Recommendations The Reliability And Validity OfReports Based On The Database Demographic And Client Identifiers The Recording Of Risk Factors Recording And Entering The Data Computer Software Summary and Conclusion 100 100 102 102 103 103 104 105 106 107 109 110 110 111 112 113 113 REFERENCES 115 APPENDICES Appendix A- Individual Prenatal Risk Identification Tool Appendix B- T-ACE Questionnaire Appendix C- Client Tracking Form Appendix D- Electronic Version ofiPRIT, T-ACE and Client Tracking Form Appendix E- Sources ofFo1ate and Iron 126 137 139 146 151 Vlll LIST OF FIGURES Page Figure 1. Geographical Location ofthe Individual Sites of the POP Figure 2. Service Delivery Model for the POP Figure 3. POP Operations and Its Evaluation Figure 4. Framework for Reviewing An Electronic Database Intended For Health Program Evaluation 4 8 17 42 lX LIST OF TABLES Page Location, Name And Sponsoring Agency Of The Individual Sites Of The POP, 1997 Table 2 Comparisons Of Annual Evaluation Reports Of The POP Table 3.1 IPRIT - Physical Factors Table 3.2 Previous Pregnancy Loss And Client History Of Spontaneous Abortion, Elective Abortion And Stillbirth Table 3.3 Pre-pregnancy Weight And Client' s BMI Table 3.4 Grand Multipara And Gravida Table 3.5 Age 17 And Under/Age 36 And Over At Time ofDelivery and Client's Actual Age Table 3.6 IPRIT- Substance Abuse Factors Table 3.7 Smoking And Number Of Cigarettes Smoked Pre-pregnancy Table 3.8 Alcohol Use And Number Of Drinks Consumed Pre-pregnancy Table 3.9 Other Drug Use, Inappropriate Use Of Over The Counter And Treatment Drugs And Pre-pregnancy Drug Use Table 3.10 IPRIT - Socio-Economic Factors Table 3.11 Single Parenthood And Marital Status Table 3.12 Financial Problems And Client's Financial Status Table 3.13 IPRIT - Emotional Factors CTF - Section One: Program Information Table 4 Table 5 CTF - Section Two: Referral Data CTF - Section Three: Intake Data Table 6 Table 7.1 CTF - Section Four: Client Characteristics Table 7.2 Other Language Table 7.3 CTF - Section Four: Client Characteristics Table 7.4 CTF- Section Four: Client Occupation Information Table 7.5 CTF- Section Four: Financial Information and T-ACE Scores CTF - Section Five: Past Pregnancy Data Table 8 Table 9.1 CTF - Section Six: Date of Assessment Table 9.2 CTF - Section Six: Client Weight Monitoring Table 9.3 CTF - Section Six: Food Intake (number of servings per day based on 24 hour recall Table 9.4 CTF - Section Six: Food Intake (number of servings of each food group) Table 9.5 CTF - Section Six: Food Intake (caffeine, sweetened drinks and water) Table 9.6 CTF - Section Six: Food Intake (iron and folate sources) Table 9.7 CTF - Section Six: Cigarette Smoking Table 9.8 CTF - Section Six: Alcohol Use Table 1 3 21 50 51 53 56 57 58 58 59 61 62 63 65 66 70 71 72 73 73 74 75 75 76 78 78 79 80 81 82 83 84 X Page Table 9.9 CTF - Section Six: Illicit Drug Use Table 9.10 CTF - Section Six: Type of Drugs Used - Qualitative Data Table 10.1 CTF - Section Seven: Counselling Contacts Table 10.2 CTF - Section Seven: Other Contacts Table 10.3 CTF - Section Seven: Physician Contact Information CTF - Section Eight: Referrals Table 11 Table 12.1 CTF - Section Nine: Program Outcome Table 12.2 CTF - Section Nine: Medical Complications Information Qualitative Data Table 12.3 CTF - Section Nine: Doctor Visits Table 12.4 CTF - Section Nine: Reasons Why Breastfeeding Discontinued Table 12.5 CTF - Section Nine: Alcohol Use And Smoking Follow-up Table 13.1 CTF - Section Ten: Coping Methods - Qualitative Data Table 13.2 CTF - Section Ten: Number of Drinks Table 13.3 CTF - Section Ten: Alcohol Consumption Table 13.4 CTF - Section Ten: Prior Treatment Times - Qualitative Data Table 13.5 CTF - Section Ten: Prior Treatment Places Table 13.6 CTF - Section Ten: Personal Goals Table 14.1 CTF - Section Eleven: Smoking Information Table 14.2 CTF - Section Eleven: Methods of Cessation 84 85 86 86 87 88 89 90 90 91 92 93 94 95 95 95 96 97 98 XI ACKNOWLEDGEMENTS I wish to express my sincere appreciation for the advice, guidance and encouragement given to me by my supervisor, Dr. David Fish. Gratitude is also extended to the other members of my supervisory committee, Dr. Bruno Zumbo and Professor Annette Browne. Thank you to Janice Linton and Lee Fischer of the Ministry of Health for the provision of and assistance with the data for this research project. I also appreciated the assistance provided to me by Dr. Shucai Guan in the transferring of the data and to Xiaolong Yang for the assistance he provided in the analysis of the data. I would particularly like to thank Liz Neave of the Quesnel and District Child Development Centre for allowing me to use their computer services in the write up of this thesis . The support and encouragement from family and friends helped me tremendously to complete this project and for this I am eternally grateful. Heartfelt gratitude must be extended to Scott Miles for helping me see this project through to its completion. XII DEDICATION I would like to dedicate this thesis to my "second mom" and aunt, Doris Brown (April12, 1922 to November 13, 1996). Chapter 1 INTRODUCTION A number of high risk prenatal counselling programs in both Canada and the U.S. aim to prevent low birth weights. In the United States, Women Infant and Children Programs (WIC) are available nation-wide and their main purpose is to provide nutritional assistance to mothers and their children. In British Columbia (BC), the Healthiest Babies Possible Program, funded by the City of Vancouver, was initiated in 1975 and was the model used for the development of the provincially funded Pregnancy Outreach Program in 1988. The Pregnancy Outreach Program (POP) is one component of the Government ofBritish Columbia's effort toward the prevention oflow birth weight and Fetal Alcohol Syndrome infants. These adverse pregnancy outcomes are considered to be among the most pressing issues in prenatal care in Canada since resource requirements for these infants in terms of neonatal intensive care and continuing health problems are considerable (Cohen & Mac William, 1995). The Pregnancy Outreach Program OfBC The Pregnancy Outreach Program of BC (POP) provides health counselling and peer support to high risk pregnant women who do not typically access traditional prenatal health services. Funding for POP is provided primarily by the Ministry of Health and awarded by contract to sponsoring community agencies. Each site is staffed by nurses, nutritionists, outreach workers, dental hygienist consultants and volunteers. A health professional (either a registered nurse or a registered dietitian-nutritionist) coordinates the Program, while peer counsellors are the primary service providers. 2 The POP began in 1988 in BC with the funding of eight community pilot sites: Cranbrook, Duncan, Nanaimo, Port Albemi, Prince George, Surrey, Terrace, and Williams Lake. In 1991 /92, based on the success of the eight pilot sites, the Program was introduced in Campbell River, Kamloops, Quesnel, Smithers and Victoria (Esquimalt). One year later, the Ministry of Health allocated monies from the provincial budget to enhance funding to the fourteen existing sites and to expand the program to eight new sites which included Burnaby, Delta, Fernie, Mission, Nelson, Prince Rupert, Salmon Arm, and Ucluelet. Three of the POP sites (Fernie, Nelson and Ucluelet) which were funded in 1992 had fewer than 250 births per year. These communities were selected specifically to demonstrate that the program could be provided on a smaller scale. Since 1992, enhancement funds have been provided to individual sites based on submitted proposals and some sites utilized these monies to create satellite services from their main site. Currently, there are twenty-one communities actively offering the POP in BC. The location and names of the individual sites of POP as well as the sponsoring agencies are shown in Table 1 on the following page. Figure 1 (p. 4) illustrates the geographical location of the individual sites of the POP inBC. 3 Table 1 Location, Name And Sponsoring Agency Of The Individual Sites Of The POP, 1997 Site location Site Name Burnaby Pregnancy Outreach Program Campbell River Babies Best Chance Cranbrook Better Babies Delta Healthiest Babies Possible Duncan Cowichan Valley Healthiest Babies Possible Esquimalt Best Babies Fernie Pregnancy Outreach Program Baby's HeadStart Kamloops Pregnancy Outreach Program Mission Nanaimo Building Better Babies Nelson Pregnancy Outreach Program Port Alberni Building Healthier Babies Prince George Healthiest Babies Possible Prince Rupert Pregnancy Outreach Program Quesnel Pregnancy Outreach Program Salmon Arm Smithers Surrey Terrace Ucluelet Williams Lake Pregnancy Outreach Program Baby Project Healthiest Babies Possible Building Healthy Babies Healthy Baby Program Prenatal Outreach Program Sponsoring agency Burnaby Family Life Institute Campbell River Family Services Cranbrook Home Support Services Surrey Community Resources Society Cowichan Valley Native Friendship Centre Esquimalt Neighbourhood House Fernie Women 's Resource Centre and Drop-in Kamloops Home Support Services Association Mission Community Services Society Tillicum Haus Native Friendship Centre Nelson and District Home Support Services Society Family Health Centre, Health Outreach for Parents and Infants Association Northern Family Health Society Friendship House Association of Prince Rupert Quesnel and District Child Development Centre Shuswap Home Support Services Society DZE L K'ANT Friendship Centre Surrey Community Resources Society Terrace Child Development Centre Nuu-chah-nulth Health Board Cariboo Friendship Society 4 • *A Plot Sites (19881811) Addltlon•f Sit• (111111192) Addltlon.t Sites (1992193) Smo1hers * Southern Vancouver Island & Lower Mainland · *Qenej ~ \J ~ 1 • ~~ • ~ \ Lak• ~ \ ~~~ * ~~~ -~~ ~ Kamooaos ~ Uduelel . : . .... : : . .. .. . . . . . . · . Figure I . Geographical Location ofthe Individual Sites ofthe POP ~ ~ ·~ ~ care ~~~ Asatmon Ann Nelso* .............,\ 5 POP Goals And Objectives The overall goal of the POP is to promote positive health practices that contribute to the health of mothers and their infants. The strategies used to achieve this goal are outreach, education, and support to clients of the program. The nine specific objectives of the POP and how they are measured are described in the following: 1. Improve Nutrition • To encourage an appropriate weight gain for pregnancy usmg standards for underweight, average, and overweight women. • To increase the number of servings from each food group to the m1mmum recommended in the "BC Food Guide for Pregnancy ." • To encourage an adequate intake of foods rich in protein, iron, calcium and folic acid, nutrients important to fetal development. • To encourage the development of a well-balanced meal pattern for the client and her family. • To improve food security by providing food supplements, meals or snacks at dropins, and/or referral to community resources as required. 2. Decrease Smoking • To eliminate or at least decrease the number of cigarettes smoked by those clients who smoke, and to maintain this behaviour throughout the pregnancy. • To reduce exposure to second-hand smoke during pregnancy. 3. Decrease Alcohol Use And Drug Misuse • To decrease the number of alcoholic drinks consumed by the client who drinks, decrease the incidence of binge drinking, and to maintain the decrease throughout the pregnancy as well as to encourage and support abstinence. • To reduce drug use to only those drugs approved by a physician. 6 • To identify women at risk for alcohol and drug abuse by using the T-ACE questionnaire and taking a history of personal and family alcohol and drug use for all clients. • For clients with identified risk for substance abuse, to ensure referral to appropriate counselling and intervention in conjunction with the local Alcohol and Drug Program. 4. Raise Self-esteem This measure is under development. 5. • 6. Encourage Breastfeeding To encourage breastfeeding during program contact in order that clients are breastfeeding on hospital discharge and continue for at least six weeks. Promote Dental Health • To identify women in need of dental care by using the dental screening questions. • For clients with identified urgent dental need, to ensure referral to appropriate treatment in consultation with the Health Unit dental hygienist. 7. • 8. Encourage Physical Activity To increase participation in some form of physical activity such as walking or swimming, to at least three times per week. Encourage Early Physician Care • To encourage the client to seek physician care, prior to her second visit to the program. • To encourage the client to attend follow-up prenatal visits with a physician. • To ensure referral to a physician for treatment and intervention to any client with an emergent medical concern. 9. Promote Social/Community Support • To encourage the participation of the client's family and friends in the program. • To encourage the client to access applicable services available in their community during pregnancy and after birth of the baby. 7 These represent the general objectives of POP. For each client of the program, the specific objectives are identified by a staff member in consultation with the client. POP Service Delivery Model The service model for the POP is illustrated in Figure 2 (p. 8). It begins with referral from sources such as community agencies or physicians. During initial contacts, the staff member determines the client's specific risk factors using the Individual Prenatal Risk Identification Tool (see Appendix A). In addition to client risk measurement, the TACE Questionnaire (Sokol, 1988) is also administered. The T-ACE (see Appendix B) consists of four questions related to identification of risk drinking. The acronym "TACE" is based on the questions which evaluate .tolerance to alcohol, ~ client's drinking may create, whether the client feels she needs to ~ - that the her alcohol intake and whether the client feels she needs to have alcohol in the morning as an ~ opener (Sokol, 1988). Eligibility for entry to the program is based on the existence of one or more risk factors indicated on the Individual Prenatal Risk Identification Tool (IPRIT): that is, only women who are identified as having one or more risk factors for adverse pregnancy are eligible for enrolment in the program. Consultation between the client and program staff results in the development of a care plan. One staff member is selected as the key worker for each client, although the program functions on a team approach to the provision of services. The four essential components of the POP are: 8 I Referral to POP I , Assessment * Risk Identificatior * T-ACE 1 Jcare PlanJ Outreach * Group Sessions * Food Supplements * Individual Counselling * Referral 1 Reassessment * Case Conferencing Referral from POP I Figure 2. Service Delivery Model For The POP 9 • Group Sessions. The individual sites are required to have drop-in sessions. These are held at least once every two weeks. Healthy foods are served and, typically, there is a presentation on client-selected topics. • Food supplements. Food supplements are a required component of the program and are provided for clients who are in financial need. Milk and juice are the most commonly offered supplements. • Individual Counselling. In keeping with the ' outreach' nature of the program, individual counselling sessions may be provided at the program site, in the client's home, or wherever the client is most comfortable. • Referral. The POP staff refers clients depending on their needs to various community resources. These include the Health Unit, Friendship Centres, YMNWCA, La Leche League and other breastfeeding support groups, Homemakers, Food Banks, Twin Clubs, the Salvation Army, Alcohol and Drug Programs, parenting programs, the local Mental Health Centre, as well as the Ministry of Child and Family Services and Ministry of Human Resources. Each POP site follows the described service delivery model, although sites may choose to allocate their budget and resources in different ways in order to respond to the particular needs of the target populations. POP Administration The Prevention and Health Promotion Division of the BC Ministry of Health has overall provincial responsibility for the POP. The Nutrition Branch of this Division coordinates the overall planning and forecasting for the Program. 10 During the history of POP (1988-97), responsibilities for individual programs have been shared at many and varied levels. In general, however, the roles and responsibilities of the key partners in most of the sites can be described as follows: 1. Sponsoring Agency A sponsoring agency provides the POP under contract from the Ministry of Health. The sponsoring agency is selected on the basis of its acceptability to the highrisk target group and its stability within the community. The sponsoring agency may provide the POP service as its only mandate or it may integrate the service with other services within its organization. The sponsoring agency for each of the twenty-one POP sites in British Columbia was shown in Table 1 (p. 3). 2. Local Advisory Committee The local advisory committee to the individual POP site has a mandate to address issues raised by that particular program. Membership of local advisory committees may include representatives from the local health departments, Ministry for Children and Families, alcohol and drug programs, sponsoring agencies, physicians, and hospital staff. 3. Local Health Department and Alcohol and Drug Programs The Health Unit/Department, specifically public health nurses, nutritionists, and dental hygienists, provide a supportive role to the local POP. The Provincial Alcohol and Drug Programs offer a variety of direct and supportive services to the individual sites of the POP. 4. Provincial Advisory Committee The Provincial Advisory Committee is responsible for ensuring the continuity and integrity of the Program. The membership of the committee reflects the provincial 11 interagency commitment to the program and includes prenatal policy and program experts from: Alcohol and Drug Programs, Ministry of Health; British Columbia Fetal Alcohol Syndrome Resource Group; Family and Children's Services, Ministry of Social Services; Family Health, Ministry of Health; Healthiest Babies Possible Program, Vancouver Health Department; and Native Health, Ministry of Health. The Concept Of Comprehensive Prenatal Care Pregnancy outreach is an example of a service that addresses health issues in ways that go beyond the traditional medical management approach to prenatal care. Since the mid 1970's, there has been an emphasis on prenatal programs that address social and behavioural as well as medical issues. Such programs have been described by the Department of Health and Human Services Low Birth Weight Prevention Work Group's Expert Panel on Prenatal Care (1986) as: consisting of health promotion, risk assessment, and intervention linked to the risks uncovered ... requiring the cooperative and coordinated efforts for the woman, her family and her prenatal care providers ... beginning when conception is first considered and continuing until labour begins ... with objectives relating to outcomes through the first year following birth (Mortimer et al., pg. 783). Comprehensive prenatal programs are considered to be multidimensional with foci on pregnancy and labour education, lifestyle behaviour modification, nutrition education and psychosocial support, in addition to traditional medical care. Thesis Focus Systematic evaluation of the POP is central to the maintenance and continued funding by the provincial government. In order to evaluate the program, each site collects data on client characteristics, risk factors, obstetric history, and birth outcomes. 12 The data is reported in a systematic format, using an electronic database. It was intended initially that this thesis would use the provincial database to investigate whether there was a relationship between the identification of risk factors and the actual birth outcomes. Before developing the epidemiological model that would test these relationships, however, preliminary analyses of the database were undertaken to determine the feasibility of doing such an analysis. In the course of the preliminary analysis, it was found that, not only was the data difficult to access but that missing values, incomplete records, and various inconsistencies in recording, precluded undertaking the proposed epidemiological analysis. It was clear that such an analysis of the database would not reflect the experience of the population of the clients in the program within a specified time period and that there were limitations to the way in which the variables were measured. Thus, the proposed analysis that was to be undertaken would not contribute to an understanding of the relationship between the various predictors and the outcomes with reliability and validity. It was agreed, therefore, that since the feasibility study had placed the database under close scrutiny, the focus of the thesis would be on evaluating the database itself. The thesis would identify the problems that were occurring in the recording of the data on individual clients at the local program level. The problems that were occurring in the transference of the data in the various client records to the electronic database would also be addressed. It was considered that such an evaluation could contribute to enhancing the way in which data is collected and recorded in electronic form, thus making it accessible and useful for systematic evaluation and epidemiological analyses. 13 Therefore, this thesis is concerned with the evaluation of the POP database that is derived from the client charts at the individual sites. In Chapter Two, a detailed description of the evaluation of the POP is given. Chapters Three and Four provide a literature review of risk identification and issues related to health program evaluation. Chapter Five describes the methodology used for this study, while Chapters Six and Seven provide the study results. recommendations of this study. Chapter Eight summarizes the conclusions and 14 Chapter 2 THE PROVINCIAL EVALUATION OF THE PREGNANCY OUTREACH PROGRAM Evidence of the success of the BC Pregnancy Outreach Program (POP) is required to justify its services and continuing support. The provincial evaluation of this program is intended to assess the impact of the interventions on the clients and to address the challenges that arise among the individual sites due to program variation between the sites and differences in client characteristics. It is also intended to address the issue of the difficulty in measuring the impact of the program on pregnancy and birth outcomes. The evaluation of the POP was planned by the Provincial Advisory Committee at its inception and involves reporting of data collected by each program. The evaluation has been concerned primarily with three areas: implementation, effectiveness, and with acceptance and satisfaction by the clients. The implementation questions that have been addressed in the evaluation process include: • Are the individual sites reaching their target groups? • Are the programs reaching clients early in pregnancy and retaining them long enough to have an impact on pregnancy outcome? • Are programs being integrated with existing services in the community? The effectiveness of the program is intended to be evaluated by assessing changes in the reported health behaviour of clients. Information about health behaviours is collected 15 and recorded about the client at intake and at the last visit prior to the delivery of their baby. The forms used to collect the information are the Individual Prenatal Risk Identification Tool (IPRIT) and T-ACE Questionnaire. Both of these forms are used as a basis for determining eligibility for enrolment. In addition, a client tracking form (CTF) (Appendix C) is completed while the client is with the program. Information from each CTF is entered into an electronic version that was developed using a standard software program (Appendix D). An evaluation of the acceptance and satisfaction with the POP was undertaken in the Spring of 1990 and culminated in the Qualitative Evaluation Report (BC Ministry of Health, 1993). Other studies of the POP have been conducted in addition to the provincial evaluation. These have included a case-control study conducted in 1995, a study that was planned to measure the impact of POP on maternal and infant outcomes. Individual sites of the POP used for this study included those located within the Central and North Vancouver Island Health Regions (Martin & Armstrong, 1995). In addition, a process evaluation in 1995 was undertaken to investigate the nutrition component of the POP (Code, 1995). Organizational Principles Of The POP Database In 1992/93, a contractor was hired by the POP to revise the evaluation forms and data collection systems to improve the ease of generating statistical reports . Using Epi Info Software (Dean et al., 1990), the contractor developed a data entry program that could be used by each site to input its own data and generate the required reports in a standardized format across the province. 16 The overall intent of the development of the POP database was to measure program goals and objectives. The following principles also governed the establishment of the POP electronic database system: • it is based on the POP client population. • it records the risk factors of the clients that are used to determine eligibility for POP enrollment. • it describes intensity ofuse and differential use across individual sites. • it is relevant to program staff and stakeholders. • it has the ability to create location and annual profiles. • it can provide for program-related research. • it can be used for administrative purposes including the assignment of funds to individual programs, to support requests for funding from legislatures and other granting agencies, and for the planning of services. The information system is designed to assess the health status of the client population using a variety of indicators, as well as to assess risk characteristics that form the basis for program intervention. The system can track POP use by clients across the province. The information system is organized around issues relevant to policy-making, program evaluation, program administration, and research at both the local and provincial level. Management Of Client Data For The Provincial Evaluation An overview of the data collection, data entry and analysis processes for the provincial evaluation as it operates at the individual sites of POP is illustrated in Figure 3 on the following page. Since eligibility for the program is determined by risk factors, all 17 Unable to contact client. IPRIT and Client Tracking Form completed as fully as ossible Client does not start program. IPRIT and Client Tracking Forms completed as fully as possible. Client starts program. IPRIT and Client Tracking Forms updated continually. Program discharge. IPRIT and Client Tracking Forms fully completed. Step 4 Data Entry Entry ofiPRIT and Client Tracking Form information into Epi Info Softwar Program by staff. Program data sent to Ministry of Health for analysis Figure 3. POP Operations And Its Evaluation 18 clients of the program are assumed to be at-risk for having a low birth weight (LBW) baby or other adverse pregnancy outcomes. The risk factors are measured by the IPRIT, an instrument that is based on areas that include physical, socio-economic, emotional and substance abuse factors. It is recommended that participants be no more than twentyeight weeks pregnant at intake, to allow sufficient time for intervention to be effective. However, clients who are identified as being at-risk for substance abuse as measured by the T-ACE Questionnaire can be admitted at any time in their pregnancy. Local advisory committees have the discretion to set local criteria to determine whether women over twenty-eight weeks gestation who do not have an alcohol or drug risk can be admitted to the program. Each program develops its own system for collecting the required client information for the provincial database. Client data derived from the charts is subsequently recorded on both the IPRIT and CTF (BC Ministry of Health and Ministry Responsible for Seniors, 1995). This information is then either entered in the electronic database at the site or the forms are submitted to the Ministry of Health and entered into the provincial electronic database there. The data are submitted for every woman who is referred to the program and whose due date falls during the fiscal year reporting period (e.g. April 1, 1994 to March 31, 1995). The data submitted to the Ministry ofHealth include women who were: • referred to the POP but were not assessed. This may occur because the client could not be located to be encouraged to enrol in the program. 19 • assessed by program staff but not enrolled. A client who is not at risk or does not wish to access POP services would not be enrolled. • enrolled into the program but did not stay in the program to delivery or end of pregnancy. A client may have moved or decided to discontinue accessing services of POP once enrolled. • enrolled into the program but did not have five or more program contacts. Provincial standards of the POP indicate that a pregnant woman must have had at least five contacts with program staff in order to be considered a client who completed the program. • completed the program. To complete the program, a client must stay in the program to delivery or end of pregnancy, and have greater than or equal to five program contacts with staff members. Information on all forms is to be completed by program staff, given the extent of a woman' s involvement with the program as outlined above. If information for a particular item on the computerized questionnaire is not available the field is to be filled with "9's" (e.g., 999999 would be entered for a six character field). Epi Info Version 5 Software Program Epi Info Version 5.0 was the software program selected for the POP evaluation. The Epi Info Program was developed for epidemiological investigation, but databases can be formatted in this software to audit other related topics (Hollyer, 1991). There are three levels in Epi Info for processing data derived from questionnaires and other structured formats. At the first level, the data gathering form can be formatted for 20 computer-assisted data entry. For the POP database, the IPRIT and CTF are available in Epi Info in this format. The next level of this software program includes the shaping of data entry and analysis. This includes producing lists, frequencies, and cross tabulations. Records may be selected or sorted based on defined variables, "if' statements, and mathematical and logical operations carried out. Graphing, formatting of reports, generation of new data sets, and a programming language are included. Databases can be linked and analyzed. The third level of Epi Info refers to the establishment of a permanent database system. This includes programming the data entry process, specifying the format of reports in customized tables, entering data into more than one file during the same session, and linking different types of files. The data entry process was pre-programmed for the POP in Epi Info and has been developed to provide a permanent and ongoing database. The POP database provides the basis for evaluation at both provincial and local program levels. The purpose of the database program at each individual site is to allow individual sites to compile, store, and generate reports on their own client population (Fairburn & Dhanani, 1993). Status Reports The POP prepares an annual status report based on the data collected each year. The report is intended to focus on implementation issues as well as the impact of program intervention on the clients. Table 2 provides an overview of the content of previous Quantitative and Qualitative Status Reports produced since the POP began. 21 Table 2 Comparisons Of Annual Evaluation Reports Of The POP Information reported Fiscal year Type of evaluation 1989 to 1990 Quantitative 1. Implementation Indicators: Characteristics of participants, risk Evaluation factors , timing of intervention , client retention , intensity of service, Report referral sources, program referrals 2. Outcome Assessment: Nutrition, smoking, alcohol use, drug use, emotional support, encourage breastfeeding 1991 to 1992 Status Report Service Delivery Indicators: Client retention, referral sources, characteristics of clients, client risk factors, timing of intervention , intensity of service, referrals from the program 2 . Client Outcome Indicators: Nutrition, smoking , alcohol use, drug use, social support, encourage breastfeeding, birth outcomes 3. Three Year Comparisons : Client load, client retention , sources of referral, service delivery, nutrition, smoking, alcohol use, birth outcomes 1992 to 1993 Status Report Service Delivery Indicators: Client retention, referral sources , client characteristics, client risk factors, timing of intervention, intensity of service and program referrals 2. Client Outcome Indicators: Nutrition, smoking, alcohol use, drug use , social support, encourage breastfeeding , birth outcomes 3. Four Yearly Comparisons : Client load, client retention, referral sources, service delivery, nutrition , smoking, alcohol use, birth outcomes 4. Comparisons of Client Characteristics 1992/93: Characteristics of those clients who completed the program as compared to those who enrolled but did not complete, service and intervention , comparative characteristics of clients by completion of program , completion rates in relation to client risk factor, completion rates in relation to sources of referral , nutritional comparisons 1993 Qualitative Evaluation Information collected by client interviews to assess the impact of the program on clients and their families and friends 1993 to 1994 Status Report 1. Service Delivery Indicators: Client retention, sources of referral , characteristics of clients, client risk factors, timing of intervention , intensity of service, referrals from the program 2. Client Outcome Indicators : Nutrition, smoking, alcohol use, drug use, social support, encourage breastfeeding, birth outcomes 3. Yearly Comparisons: Client load , client retention , sources of referral , service delivery, nutrition , smoking , alcohol use, birth outcomes 1994 to 1995 1994/95 Annual Report 1. Program Goals : Nutrition , smoking, alcohol and drug use, encourage breastfeeding 2. Birth Outcomes 1. 1. 22 Summary Evaluations have become the norm for most publicly funded social programs in the 1970' s and such mandates remain a strong impetus for current health program evaluation work (Stecher & Davis, 1987). POP has evaluated its program since its inception and this has been continued on an annual basis. Against the background description of the POP database that has been presented in this chapter, the next chapter reviews selected literature related to prenatal intervention programs. It first reviews the literature related to prenatal scoring systems in order that the criteria used by the POP and recorded in the database can be related to a broader experience. The literature related to the evaluation of prenatal programs is then reviewed in order to place the potential of the POP database in a wider context. 23 Chapter 3 LITERATURE REVIEW - RISK FACTORS AND RISK SCREENING In this chapter, the literature related to risk assessment in pregnancy is reviewed. This chapter is subdivided into three sections. The first section reviews literature relevant to risk scoring systems. A majority of the current systems of risk assignment in pregnancy discussed in the literature relate to hospital and physician records. The POP uses a risk scoring tool, the Individual Prenatal Risk Identification Tool (IPRIT) to determine client eligibility instead of using it in a clinical context to assess risk of preterm labour and adverse birth outcomes. Although the POP has a somewhat different focus for the use of risk scoring, it was decided that in order to critically examine the data collected by the IPRIT, a review of other scoring systems and the literature related to them was warranted. The next section reviews the literature related to risk factors for adverse pregnancy outcomes. Most risk scoring systems share risk factors in common with other risk factors being included or excluded depending on the primary purpose of the risk scoring. The final section discusses risk scoring systems as they relate to prediction of pregnancy outcomes. Risk Scoring Systems Risk assessment refers to the identification of clients who are more likely to have adverse birth outcomes. Risk assessment is typically conducted at the first visit and continues during each prenatal encounter. A variety of systems of assessment are used, with each system covering factors that are both obvious and subtle ip their relationship to 24 birth outcome (Morrison, 1990). Women deemed to be "at risk" are those who have an increased chance of adverse birth outcome above the baseline of the 4-6% noted in "normal pregnancies" (Morrison, 1990). Risk factors are attributes statistically associated with the risk of a given adverse outcome. Risk factors may be related to the characteristics of the individual woman, her environment or her treatment. The adverse outcomes include maternal and child mortality, morbidity, and functional impairment. A risk factor for a given outcome may be a causal determinant of that outcome or it may merely be a predictive "marker" that a negative outcome is more likely (Backett, 1984). Studies have shown that risk assessment improves risk detection (Essex & Everett, 1977; Dissevelt, 1976) and that it is a valuable tool for the health care provider who is responsible for those at risk (Wilson & Schifrin, 1989; Lesinski, 1975; Goodwin et al. , 1969). Risk-scoring systems have been criticized as being characterized by low specificity and low sensitivity and some reviews have suggested that they have generally shown little benefit (Goldenberg et al. , 1990; Heins et al. , 1990; Maine et al. , 1989; Konte et al. , 1988; Herron et al. , 1982). Hueston (1992) pointed out that the high risk scoring system tends to identify clients who are at high risk for medically indicated iatrogenic preterm delivery or premature rupture of membranes but that the scoring system has poor positive predictive value when used in lower risk populations. Others (Maine et al. , 1989) have indicated that risk scoring systems vary in predictive value since they may increase the chance of unwarranted health intervention in pregnancy. They also carry the inherent disadvantage of stigma and anxiety associated with "high risk" labeling (Creasy, 1993 ; Maine et al. , 1989). 25 Although risk scoring has been reported to correlate with perinatal outcome, the validity of these correlations is questionable when subjected to strict epidemiological analysis (Health and Welfare Canada, 1981). It has been suggested that rigorously designed studies have yet to be published that test the hypothesis that risk scoring alone modifies perinatal outcome. Most of the systems give, at best, a positive predictive value of only 20-35% and a sensitivity of 40-70% (Creasy et al., 1990). In fact, several studies have found that information collected during a single antenatal visit on variables such as age, parity and obstetric history, together with measurements of height and weight was sufficient to identify the majority of risk factors (Shah & Shah, 1981 ; Lennox, 1981 ; Shah, 1978; Hart, 1977; Essex, 1977; Dissevelt, 1976). Investigations of risk scoring systems have revealed that there are difficulties associated with the use of prenatal risk forms for accurate data collection. These difficulties include completion compliance and validity of recorded data. For example, a survey of over 2000 prenatal records (Canadian Task Force on the Periodic Health Examination, 1979) showed a wide variation on an item-per-item basis as far as completion rates. Some items had a completion rate of greater than 80%, others less than 50%. Any data with this much variation produces inaccuracies. With respect to the validity and reliability of the data recorded, the risk identification forms often ask for information that is inadequate for predictive purposes. For example, risk assessments often require information on smoking in the form of a 'yes' or 'no' answer. This question should include the type of cigarette, the amount of nicotine contained in this brand, the number of cigarettes smoked, the period of gestation during which smoking continued and if and when the client stopped smoking during pregnancy. 26 Risk Factors Although many risk scoring systems are comprehensive, the inclusion of some risk factors and the exclusion of others is frequently questioned. Factors that are questioned include employment, weight gain, genetic factors and diet. For example, some risk score sheets include employment during pregnancy as a risk, whereas others do not because the literature reflects evidence of both the harmful and beneficial effects of work on pregnancy outcome. The literature supporting risk identification in pregnancy reflects inconsistencies and controversial views. For example, a summary of the Women, Infant and Children Program (U.S. Institute of Medicine, National Academy of Sciences, 1992) indicated that inadequate nutrition encompassed more than just the consumption of foods consistent with the U.S. food guide and should include factors such as food security. The research literature relating to the risks associated with weight gain in pregnancy reflects inconsistent predictive values. In particular, the criteria related to timing of weight gain necessary to prevent fetal problems is controversial (The Institute of Child Health, 1993). Further, weight gain in pregnancy seems to vary among different ethnic groups (Worthington-Roberts & Klerman, 1990) and this needs to be taken into account. Many risk scoring systems include established genetic diseases as risk factors. Whether this should be included on risk score sheets that are non-medically based remains controversial. For example, programs such as POP highlight lifestyle risk factors and the identification of medical factors may be beyond the scope of the program. Such programs, however, may provide support and education to the expectant mother who would be experiencing stress as a result of genetic or medical risk factors. 27 It is apparent that there is a need for more research to fully understand predictive values of the risk factors that are currently used. Given that the understanding of the pathophysiology of adverse birth outcomes is incomplete (Creasy et al. , 1990), it is not surprising that risk scoring systems with an acceptable degree of predictive discrimination have not been developed. Pregnancy Outcome Risk factors may be specific to the pregnancy outcome to which they relate. Although overlap may occur, attributes associated with a higher risk of one outcome may not be associated with a higher risk of other outcomes and may even confer protection against them (WHO, 1994). Attributes such as young or advanced maternal age, primiparity, high multiparity, short stature, low pregnancy weight-for-height, poor gestational weight gain, close spacing of pregnancies, history of adverse outcome in previous pregnancies, severe anaemia, and cigarette smoking are often discussed as if they represent "universal" risk factors , in the sense of being associated with all, or at least most, adverse outcomes of pregnancy. However, research findings to date do not generally support this notion (WHO, 1994). For example, a woman who smokes appears to be somewhat protected from developing pre-eclampsia but her fetus is at increased risk of both growth retardation and preterm birth. According to Creasy et al. (1993), a proportion of all pregnant women, probably less than 25%, have an increased risk that require(s) them to be managed differently from the remainder. Women's pregnancies have been more appropriately managed and their perinatal death and morbidity rates reduced as a result of application of a wide spectrum of interventions by health care teams. It has been increasingly difficult to ascertain 28 whether improved pregnancy outcomes have been due to improvements in the health care system or the development of risk scoring systems. Nonetheless, risk detection in pregnancy 1s considered a valuable concept, although, its applicability may be limited to certain settings. Traditionally, such systems were used in acute health settings to identify prospectively those women most likely to deliver before 37 weeks gestation. Such risk scoring systems conduct the assessment during the first trimester and at 26 to 28 weeks gestation (Creasy et al., 1980). Studies have shown that risk assessment in the first trimester will typically identify one-third of the general population as being high risk (Main et al., 1985). Rescreening later in pregnancy may identify an additional group of women likely to be delivered before 37 weeks gestation (Main et al., 1985). However, it is questionable whether further screening would be of additional benefit especially since the more advanced the pregnancy the less the time to improve outcome by modifying lifestyle risk factors. The function of risk identification forms must be clearly defined. These forms are typically used both as a working guide for staff in their management of the pregnancy and as a form designed for accurate data collection about a program. Attempts to design a risk assessment form that achieves both of these goals often result in forms that do not satisfy fully either objective. It is apparent that, although there are many factors which have risks related to adverse outcomes in pregnancy, there is urgent need for further research to determine the predictive values of these risks and the values of the interventions that are intended to reduce the risks. 29 Summary This chapter has provided an overview of risk scoring systems, the factors that make up these systems and their predictive value. The following chapter discusses the literature related to the evaluation of intervention programs for pregnant women. 30 Chapter 4 LITERATURE REVIEW- PROGRAM EVALUATION This chapter is concerned with the health program evaluation process. The Pregnancy Outreach Program (POP) requires that data be collected and entered into an electronic database in order to provide a basis of evaluation for the POP. A review of selected literature relevant to health program evaluation places the POP evaluation process in a broader context. Evaluations Of Outreach Programs For High Risk Pregnant Women Although studies suggest that programs targeted at reducing adverse birth outcomes are well accepted by clients (Hueston, 1995; Creasy, 1993 ; Papiernik et al. , 1986), evidence demonstrating the effectiveness of these programs has been inconsistent. Early studies of preterm-birth prevention educational programs generated enthusiasm for this low-cost procedure since studies using historical controls demonstrated reductions in low birth weight (LBW) and preterm delivery in clients who were provided with education programs throughout the third trimester of pregnancy. However, a meta-analysis of studies describing the effectiveness of these programs concluded that preterm-birth prevention educational programs appeared to have little benefit in reducing preterm birth (Hueston et al., 1995). The delivery of specific interventions for high risk pregnant women has been studied primarily with smoking cessation and diabetes management. Less documentation exists regarding the efficacy of risk interventions designed to improve outcome such as LBW. While strategies for the management of high risk pregnancy interventions exist, there is 31 no definitive evaluation of the role and effectiveness of risk scoring in pregnancy m determining outcomes. Research has also shown that prenatal outreach programs or free prenatal care reduce perinatal morbidity (Moore et al. , 1986), improved access to prenatal care and birth outcomes (Schlesinger & Kronebusch, 1990; Corman & Grossman, 1985; Norris & Williams, 1984 ), and were cost effective (Moore et al. , 1986; Institute of Medicine, 1985). Although the exact mechanisms(s) through which these programs impact on infant health is not known, studies suggest that improved nutrition, preterm delivery education and screening of risk factors that arise during pregnancy (Institute of Medicine, 1988) are avenues through which prenatal care for low income women improve infant outcomes. Variability in the outcomes of these programs is due to many reasons. Differences in the interventions, application of the program to different client populations, samples being limited to high-risk clients who often deliver prematurely, and varymg methodologies have all contributed to the confusion. Many studies have used small sample sizes, limiting their power to detect small but important differences between intervention and control groups. Also, some studies have focused on intermediate outcomes, such as cervical dilation and potentially preventable preterm birth, which are not clearly linked to outcomes of LBW, preterm delivery, or neonatal survival. Because such programs are inexpensive and low risk, and because of the disproportionate increase in morbidity and mortality, and the high costs associated with preterm birth (Gold et al. , 1987), a small effect may be noteworthy. 32 Assessment Of Health Program Databases A database is a collection of data that has been aggregated and can be organized to provide useful information that can readily be extracted from the database (Ferri et al., 1993; Wiederhold, 1981 ). The concept of a database encompasses the data itself, the hardware used to store the data, and the software used to manipulate the data. The development of health program computer databases began in the 1960' s. University and government administration had databases that were used in the areas of health care delivery, evaluation of clinical interventions and more recently in areas of public health. Databases may be subdivided into macro and microdata components. Macrodata refers to the entire dataset and has many applications that include health-care management, program planning and evaluation, health policy development and health research. Microdata describes events at a local or elementary level and are generally used by health managers (Ferri et al. , 1993 ; Wong, 1984). In the context of the present study, the POP database program is an example of microdata that is intended to provide the program coordinator with information about the client population. The corresponding macrodata is the provincial database that can provide information on different variables for the entire POP clientele. Data that is required to carry out the functions of health services evaluation comes from a variety of sources and in a diversity of forms, but the data must meet standards of validity and reliability if the evaluation is to be of value (Feinleib, 1993). The following reviews the basis for the evaluation and assessment of health program databases. It groups the criteria under four headings: utility for program evaluation, utility for decision 33 making and policy development, utility for health research and selection of information system. The Utility Of The Database For Program Evaluation. In reviewing the literature, eight dimensions relevant to the assessment of health program evaluation have been described and are discussed in the following. i) Relevancy and Specificity. Can one obtain the data that are needed? Will surrogate measures or already existing data serve to make current decisions? Conversely, are scarce resources being expended on collecting data that are no longer relevant to the issues? Could collection of the data be discontinued with little loss to the advancement of knowledge? ii) Coverage. Can one gather adequate data for the population subgroups of interest? Data for local areas, small demographic subgroups, and special constituents may be costly or impossible to collect. iii) Quality. How does one determine whether the quality of a database is good or bad? The goal is to obtain the best data possible but when compromises must be made, to what extent can quality be sacrificed? Agencies that collect data must constantly judge the trade-offs between quality, costs and timeliness. Data must be of sufficiently high quality to add to the body of knowledge and have direct applicability to the decisions being made. It is not infrequent that researchers find data presented in a form that does not allow for comparability or easy use for the studies under consideration. iv) Acceptability, Collection and Dissemination. Are the data collection methods acceptable in terms of design and costs? Are the intended respondents or providers of 34 data (individuals, institutions, government agencies) able and willing to provide the information? Ethical considerations, confidentiality concerns, respondent burdens, and conflicting priorities may hamper the ability to collect the data. Will the results be accepted by users as valid and credible? There are numerous examples in which poor decisions were made because the available data were inadequate in these respects. The effectiveness of databases is derived from the fact that from one single, comprehensive database much of the information relevant to a variety of organizational purposes may be obtained (Weiderhold, 1981 ). v) Timeliness. How recent do the data have to be? How long do time series have to be to disclose temporal patterns and progress toward objectives? The time frame for responding to specific planning problems is often lengthy. Software programs have been developed for processing this data but specific problems often arise which cannot be handled through standard processing. Mechanisms and processing strategies for responding in a timely manner to such problems need to be devised. The relevance and importance of data to health program planning is a function of time. In many cases, data is out of date before it has been analysed. vi) Confidentiality. With the renewed public interest and involvement in the issue of data privacy, restrictions in accessing data files have become more specific and limiting in some instances while making data more accessible in others. More clarification of what constitutes private, confidential ~ public data needs to be forthcoming in order to alleviate the difficulties involved in accessing data for health planning purposes. 35 vii) Cost-Effectiveness. Cost-effectiveness is a central dimension for program evaluation and it is one that is not often accounted for. It considers information on the amount and complexity of the inputs, expressed in monetary units, and the outcome expressed in some composite measure. The cost of the evaluation will be determined by the complexity of the interventions and the efficiency of the resources. viii) Appropriateness of Evaluation Paradigm. Databases developed for the intent of health program assessment must be based on a specific evaluation paradigm. The evaluation process can be conceptualized as consisting of three main steps: setting the evaluation agenda; formulating and implementing the research; and termination, dissemination and use of results. Various models of evaluation have been proposed by a number of theorists or practitioners. Those models differ in their conceptions as to what evaluation is, what the relationship with the primary client and other stakeholders should be, who should be making the relevant value judgments regarding the program, and the criteria for judging the evaluation study itself. According to Bobadilla (1992), there are over eighty approaches available to evaluate maternal health programs. The three main paradigms governing evaluators and evaluation models (Hamilton, 1993 ; Ross, 1991 ; Smith & Glass, 1987; Cronbach, 1982) form the focus ofthe present discussion. The first paradigm considers evaluation to be synonymous with applied research. Rigorously designed comparative studies, true field experiments, randomized clinical trials, quasi-experiments and experiments are the methods sought. Methodological rigour, particularly internal validity, is seen as important for discerning causality. The evaluation is primarily sumrnative, comparative and quantitative. One of the most important assumptions of this paradigm is that the experimentally controlled comparison 36 provides the most valid evidence that the program produced results. Program goals must be few in number, non-problematic and clearly specified. The evaluation is targeted at one primary official policy maker who presumably acts rationally when provided with experimental data. The second paradigm conceives of evaluation as part of systems management, aiding managers in their administration of the program. The organization is seen as a system of inputs, process, and outputs. The evaluator describes these and relates them to each other. The manager can then make decisions to regulate and improve the functions of the system. Research methods tend to be surveys of decision makers to determine program goals and their information needs, client satisfaction surveys, cost analysis and, sometimes, monitoring of program processes. The evaluator is interested in level of attainment on performance indicators of the given goals, and in discrepancies between the stated objectives and performance. Some authors refer to this model as an objectivesbased or goal attainment paradigm. The final program evaluation paradigm emphasizes that valuation and politics are inextricably mixed. Evaluation research studies are not directed just at one all-powerful decision-maker, but should consider all major stakeholders who may play a role in maintaining, modifying or eliminating the program. These stakeholders in turn should be appropriately informed of the results of the evaluation. The paradigms, derived from the various writings of evaluation theorists, illustrate the scope and variety of ideas about what program evaluation is. The common dilemma for program evaluators is that it is not possible in one study to maximize comprehensiveness, 37 relevance and scientific rigour. A series of related studies gives greater flexibility , and lends itself readily to evaluation of health action. Utility For Decision-Making And Policy Development. Aikin (1969) described program evaluation as a process of ascertaining areas of concern and selecting appropriate information in order to report summary data useful to decision makers. In order to answer the questions of stakeholders, certain criteria must be considered. i) Accessibility. Are the data available to those who need them? Despite the wealth of information that can be potentially available relationships are such that the information is often not accessible. Data tends to be collected in an unstandardized manner, is closely guarded by interest groups, and is often lost in the myriad of agencies involved in the collection process since there is no central repository or communication mechanism. ii) Types of Decision Making. The literature describes four types of decision-making in evaluation: metamorphic, homeostatic, incremental and neomobilistic (Cronbach, 1982). Metamorphic decision-making is intended to produce a complete change in a system. Homeostatic decision-making is a common occurrence and aims to maintain the normal balance in a system. These changes are usually small and remedial and are used to correct deviations found at the time of evaluation. Incremental decision-making involves shifting the program to a new normal balance based on small serial improvements. Neomobilistic decision-making involves innovative solutions for significant problems. The evaluative process selected will be the basis of the type of decisions to be made. Selection Of Information System. The literature review indicated three key areas relevant to the assessment of the information system selected as the avenue for 38 documenting the information with a health program database. These are summarized in the following. i) Usability . Are the data in a usable format? Are they accompanied by appropriate software in order that the user can generate summaries, tabulations, graphs, and other analyses appropriate to the user' s needs? Often, varying units of aggregation or analysis are employed in the assembly of data. In many cases, it is difficult if not impossible to convert data from one of these aggregations to another. Traditionally, secondary data is not uniform either cross-sectionally or historically. Secondary data sources are also very uneven in validity and reliability due to methodologies employed in collection and processing of data. Secondary data, particularly that related to specific organizations and institutions, may only be made available in the aggregate. These data may be useful in uncovering trends and the possible generation of hypotheses, yet too general for testing those hypotheses. Thus, descriptive results may emerge which lack the specificity necessary to contribute to further knowledge about a particular subject. One of the key issues in assessing usability of an information system is looking at the approach of processing. Data can be processed by distributed and centralized means and this is dependent on the type of information and resources available. ii) Coordination with Existing Database Systems. Provincial and federal levels of government have identified the collection and analysis of data as a major responsibility which includes data concerning the status (and its determinants) of the health of the Canadian population as well as health care delivery systems, the effect the area's health care delivery system has on the health of the population, the number, type, and location of 39 Canadian health resources, including health services, manpower, and facilities as well as the patterns of utilization of health resources in various areas. The BC Ministry of Health has taken many initiatives to develop record keeping and linkage systems. The BC Royal Commission on Health Care (Province of BC, 1991) recommended that information must be as simple as possible to collect, using standardized forms; be stored centrally; allow a patient/client to be followed through the health care system; allow measurement of the effect of health care services and allow for the measurement of the population' s health. Health care workers and managers must be involved in deciding what information is needed to ensure that it is useful. Health care researchers must also be involved to ensure that the data collected are sound. Furthermore, the provincial and federal ministries of health should agree on a consistent set of data to be collected and that the information be coordinated. iii) Cost-Effectiveness. As is the case for determining utility of a health program database for program evaluation, cost-effectiveness is central to the assessment of information system selection. Utility For Health Research. Three key areas are addressed in the research literature pertaining to the utility of a database for health research. i) Using Health Program Databases for Health Services Research. Health services research has been described as: ... an interdisciplinary activity, directly relevant to health and intended to further the understanding of the many factors influencing the delivery of health care with the ultimate objective of improving the provision of health services and making more efficient use of resources. It encompasses a wide spectrum of activities ranging from fundamental 40 research, the collection of statistical information, applied research, development, testing and evaluation, to policy analysis and long range planning. Its substantive concerns are equally broad and include the planning, organization, financing, management, use and effectiveness of health services (Institute of Medicine, 1978). In Canada, the development of health program evaluation and research in the future will likely be at the provincial rather than the national level. Nevertheless, since the provincial issues are, in most cases, the national issues, future national policy may well be developed based on the experimental programs and research findings from the provinces. Can the current provincial database systems appropriately assess the impact of health programs and be a basis for national policy decision making? ii) Research Methods. A statistical format is required to evaluate most interventions for several reasons. The evaluation methodology must be sufficiently sensitive to isolate the effects of the program. The amount of control that can be exercised by the researcher depends on the method that is selected (e.g. descriptive, correlational, quasi-experimental or experimental design). In evaluation, the evaluator needs to be sure that changes in measured behaviour can be attributed solely to the program. experimental designs are ideal. Thus, the use of experimental and quasi- In reality, however, such designs are difficult to implement in the context of health program delivery. iii) Validity and Reliability. A key problem in using secondary data is the recurrent question of reliability and validity of the data. When data is collected for one particular purpose, there is no assurance that those data will be appropriate to the particular research interest. Typically, the original researcher may have asked a question that "comes close" to measuring what one was interested in. One needs to ascertain whether the question 41 that was asked provides a valid measure of the variable that is to be analyzed within the context of the proposed evaluation. A Conceptual Framework The POP database was described in detail in Chapter Two. Using the evaluative criteria discussed in the literature review in this chapter, a conceptual framework has been developed. This is outlined in Figure 4 located on the following page. This conceptual framework focuses on assessment of an electronic database intended for health program evaluation. The first step of the assessment involves describing each variable of the database and reviewing it in terms of its utility as a unit of measure. The detailed review is the basis of the next step that involves assessing the database in terms of four inter-related criteria (inter-relation is depicted by two-way arrows): utility for program evaluation, utility for decision-making and policy development, utility for health research and selection of information system. This conceptual framework is intended to form the basis of evaluating the POP database. 42 Evaluation of an electronic database intended for health program evaluation 1 sis of database content 1 Utility for program evaluation *relevancy/specificity *coverage *quality *acceptability *timeliness *confidentiality *cost effectiveness *appropriateness of evaluation paradigm \ Utility for decisionmaking and policy development *accessibility *types of decisionmaking Utility for health research *health program databases as a basis for health services research *research methods *validity/reliability Selection of information system *usability *coordination with existing systems *cost-effectiveness Figure 4. Framework For Reviewing An Electronic Database Intended For Health Program Evaluation 43 Chapter 5 METHODOLOGY This chapter outlines the process that was undertaken to evaluate the quality and integrity of the Pregnancy Outreach Program (POP) evaluation data. Data Selection To assess the electronic information of the POP database, all variables of the database were reviewed. The focus of this review is to illustrate the problems that may be encountered by analysts who use the database for research and evaluation and to guide further development of the database. At the time of study, the POP database that was available included data from the years of 1992/93, 1993/94 and 1994/95. It was decided to review the data collected between April 1, 1994 to March 31 , 1995 . This year was selected as it was the most recent time frame available at the time of study and it included all twenty-one POP sites inBC. Formatting OfData The formatting of the data as made available presented technical difficulties which required considerable time to resolve. The data set for 1994/95 was made available on two computer disks. Attempts were made to merge the data on both disks through the merge function of the Epi Info Version 5.0 Software Program. This was unsuccessful because a "runtime" error precluded reading portions of the data from the file. This error also made it impossible to open the file in Epi Info to ascertain the source of the error. 44 To resolve this problem, the data was exported to Unix, a networking software program. The "runtime" error was identified and corrected. The files were then merged in Unix and reformatted in Epi Info. Since Epi Info has limitations with respect to analytical capabilities, it was necessary for the purposes of this review that the data be exported to other software programs. Epi Info Version 5 allows for information to be transferred to SPSS, Lotus, DBASE and SAS. An attempt was made to directly transfer the 1994/95 file to DBASE but was unsuccessful as much of the data specific to each variable merged into single columns. In order to transfer the file within the available system capabilities, the following steps had to be taken: 1. The file from Epi Info Version 5.0 was transferred to Lotus 1-2-3. 2. From Lotus 1-2-3, the file was transferred to Microsoft Excel. 3. From Microsoft Excel, the file was transferred to DBASE Version 5.0. The total number of records that appeared in Lotus 1-2-3, Microsoft Excel and DBASE files was 1741 which differed from the 1738 open records indicated in the Epi Info Version 5.0 Database File. Data Analysis Analysis of the data was conducted using both Microsoft Excel and DBASE 5.0 Software Programs. The database consisted of both qualitative and quantitative information. A total of 130 numerical and nominal variables existed in the database which were categorized based on their origin as follows: 45 1. Individual Prenatal Risk Identification Tool (IPRIT) Consists of thirty dichotomous variables subdivided into four groups: physical (twelve variables), substance abuse (four variables), socio-economic (eight variables) and emotional (six variables) factors. 2. Client Tracking Form (CTF) This is subdivided into sections that included: Initial Intake Information (four variables), Referral Data (three variables), Intake Data (six variables), Client Characteristics (twelve variables), Past Pregnancy Data (ten variables), Client Monitoring Data (fifty-two variables), Project Contact Data (seven variables), Referrals (one variable), Program Outcome (fourteen variables), Alcohol Data (fourteen variables) and Smoking Data (ten variables). Quantitative variables are dichotomous, continuous and categorical. Qualitative information derived from open-ended questions is also collected in this section of the database. 3. T-ACE Questionnaire The T-ACE Questionnaire has four questions that are significant identifiers of risk drinking. The scores on this range from zero to five with a score of two or greater being an indicator for risk drinking in pregnancy. The value of each answer to the four questions is totalled to determine the final T-ACE Score. For the POP, the T-ACE is completed at intake and the cut-off for risk drinking for the program is a score of two or greater. T-ACE Scores are also part of the CTF. Tabulations Of Each Variable The following are the criteria used to assess each variable: 46 Frequency Distributions. The frequency distribution of each variable provided a base to characterize the "quality" and reliability of the data recording. A complete record for a variable was defined as a record that contained a valid code. An incomplete record for a variable was defined as a record in which the field was left blank. A don't know was defined as a variable which had "9's" entered as the answer. Record Completion Rates. Record completion rates provided in the frequency tables were based on the number of completed records for that variable or the number of completes plus the number of specified "don't knows" divided by the total number of clients who stayed in the program until completion. Record completion rates will subsequently be referred to as "completion rates". Internal Consistency. For a number of variables, the reliability of the recorded responses for the IPRIT can be assessed by cross-tabulating the risk factors to the responses to questions in the CTF. For example, the risk factor PF12 defined as a client whose age is less than 17 or older than 36 can be compared to age of client data in the CTF. The total for most variables was based on the 1102 clients who completed the program. Some variables consisted of a subset of the client population (ie. alcohol follow-up information is based on clients with the risk factor of alcohol use indicated) and, where warranted, these are indicated as the totals for that particular variable. 47 Information Selection As indicated in Chapter Two, revision of the CTF and the IPRIT occurred in 1992 and the revised forms were distributed to all sites in the 1993/94 fiscal year. The database at this time, however, was not changed to reflect the new forms. The 1994/95 database selected, reflected the information and format of the old IPRIT and CTF and these were used as the basis of reporting the results. Chapters Six and Seven show the results of the analysis of the POP database. Chapter Six reviews the content of the IPRIT section of the database and Chapter Seven describes data from the CTF section of the database. 48 Chapter 6 RESULTS- INDIVIDUAL PRENATAL RISK IDENTIFICATION TOOL This chapter describes and evaluates the data of the Individual Prenatal Risk Identification Tool (IPRIT) as recorded in the individual client forms. Determining The Number Of Completed Files The first step of the database analysis was a count of the number of records in the computer file. A total of 2261 individual client records were identified. This number included all clients who had been referred to the program since the computerized database was initiated in 1992. In this database, the number of open records that met the definition of expected due dates between April1 , 1994 to March 31 1995 was found to be 1738. The 1738 eligible records in the data set for 1994/95 included 1102 clients who had remained with the Pregnancy Outreach Program (POP) until delivery of their baby. The remaining records in the data set included clients who were referred but did not complete the program because they were not high risk, were not interested in the program, moved, or discontinued the program. Since these records were necessarily incomplete they would not be eligible for inclusion in this investigation. The 1102 clients who completed the program represent the base for analysis in this evaluation. The Individual Prenatal Risk Identification Tool - Description The purposes of the IPRIT are to identify major factors that influence the outcome of the pregnancy and to record the risk factors specific to the individual client. The risk factors are subdivided into four categories. These include physical, substance abuse, socio-economic and emotional factors. 49 Each factor is defined in the IPRIT guide and these definitions are utilized as the basis for POP staff to record the risk information that applies to their clients. This guide, combined with experience, knowledge and intuition on the part of program staff is used as the basis for completing the IPRIT for each client. The following frequency tables presenting 1994/95 data from the IPRIT are subdivided into the four sections of the risk score sheet. Record completion rates for each variable (hereafter referred to as "completion rates") are based on the number of files recording "yes" and "no" for a particular variable divided by the 1102 clients who completed the program. IPRIT - Section One: Physical Factors Table 3.1 presents data from the first section of the IPRIT, physical risk factors. Physical risk factors refer to medical and physical conditions, as well as the obstetric history, which can influence the outcome of the current pregnancy. 50 Table 3.1 IPRIT- Physical Factors Risk factor Description Yes No Number of complete records PF1 PF2 PF3 PF4 PF5 PF6 PF7 PF8 PF9 PF10 PF11 PF12 Previous pregnancy loss Illness/Condition with impact on pregnancy Pre-pregnancy weight (BMI <19.8 or> 29) Rate of weight gain Inadequate nutrition Previous child with anomaly or disorder Previous high risk infant Multiple pregnancy Birth interval (less than 2 years) Grand multipara (fifth pregnancy or more) Established genetic risk Age 17 and under/age 36 and over at time of delivery 313 194 369 225 583 27 44 21 146 70 33 248 169 175 132 174 98 225 219 225 208 219 224 182 482 369 501 399 681 252 263 246 354 289 257 430 Completion Total rate 43 .7% 33 .5% 45.5% 36 .2% 61.8% 22.9% 23.9% 22 .3% 32.1% 26 .2% 23 .3% 39.0% 1102 1102 1102 1102 1102 1102 1102 1102 1102 1102 1102 1102 Completion rates for the risk factors in this section ranged from 22.3% for PF8 to 61.8% for PF5 . It is believed the low rate of completion for this and subsequent sections of the IPRIT occurs because the fields of the questionnaire have been left blank rather than recording a specific "n" (i.e. representing "no"). This constitutes a major problem because while an analyst could impute a "no" to the incomplete field, there are two risks in so doing. First, the blank field may indicate that this particular question was either not asked by the staff member or secondly the mother was unable to provide a definitive answer. The risk of a false negative in these circumstances, however, is such that it would be inappropriate or indeed misleading to impute a "n" where a field is left blank. This issue of missing data for these physical risk factors is also relevant to the entire section of the IPRIT. 51 In each section of the IPRIT, the risk factors will be discussed and evaluated for quality and integrity of the data. The following discusses the physical factors section of the IPRIT. PF1 -Previous Pregnancy Loss. PF1 , previous pregnancy loss, had a completion rate of 43.7%. Previous pregnancy loss includes a client's past history of spontaneous abortion, neonatal or infant death. The data collected for this risk factor is derived directly from initial intake information collected for the CTF. To verify the reliability of the data recorded for this risk factor, a cross-tabulation with related variables located in section five ofthe CTF was done. The following table shows the results of this analysis. Table 3.2 Previous Pregnancy Loss And Client History Of Spontaneous Abortion, Elective Abortion And Stillbirth PF1: Previous pregnancy loss (recorded in the IPRIT) Previous spontaneous abortion (AS), elective abortion (AE) or stillbirth (SB) (recorded in the CTF) Yes No lncompletes Total Client had one or more AS, AE or SB indicated Client had no previous AS, AE or SB indicated lncompletes Don't knows Total 295 19 10 123 8 26 0 313 169 83 183 354 0 620 397 316 388 1 1102 As illustrated by the above table, there were discrepancies between risk factor PF1 and related data collected on the CTF about the client's obstetrical history. Eighteen records which had PF1 positively indicated, did not have any prior AS, AE or SB recorded in the CTF. Conversely, of the 169 records indicating "no" for PF1 , 123 records matched AS, AE or SB information indicated in the CTF. Differences between the two 52 variables may partially be due to the lack of recording of data for infant deaths (up to 365 days of life) on the CTF. If this was the source of data discrepancy, however, it would be anticipated that the number of files indicating "yes" for the risk factor, PFI, would be greater than the total number of client files on the CTF with previous AS, AE or SB indicated. PF2 - Illness/Condition With Impact On Pregnancy. This variable had a completion rate of 33 .5%. The list in the guide is extensive but there are no clear criteria for inclusion or exclusion of the various illness conditions named. In addition, some criteria for PF2 was replicated in the IPRIT guide. For example, 'very rapid weight gain' appears both in PF2 and PF4, thus creating a problem of double counting if the risk was present. PF3 - Pre-pregnancy Weight. Considerable emphasis has traditionally been placed on weight gain during pregnancy management, however, PF3, had a completion rate of only 45.5%. This risk factor is based on the body mass index (BMI) measurement that is a ratio of one's weight measured in kilograms to the square of one's height measured in metres. Discussion with program staff of the individual POP sites revealed that how the BMI is calculated is inconsistent. Some program staff calculate it based on the established formula provided in the IPRIT guide, whereas other staff members use resources such as The Canadian Dietetic Association's Body Mass Index Calculator. These different methods will provide answers within approximately 0.5 of one another. To verify the reliability of the data recorded for this risk factor, cross-referencing was done with the BMI's provided in the CTF (refer to section six). The following table shows the results of the analysis. 53 Table 3.3 Pre-pregnancy Weight And Client's BMI PF3: Pre-pregnancy weight (recorded in the IPRIT) Client's Body Mass Index (BMI) (recorded in the CTF) Yes No lncompletes Total Prepregnancy BMI less than or equal to 289 19.8 or greater than or equal to 29 68 BMI between 19.9 and 28 .9 lncompletes 11 Inadmissible 0 Don't knows 1 Total 369 14 69 372 104 11 1 2 132 460 69 0 3 601 632 91 1 6 1102 As the above cross-tabulation indicates, there were discrepancies in the data that were collected. There were 68 records which had the risk factor PF3 assigned but the actual BMI data collected for these records in the CTF did not match the criteria of this risk factor. Conversely, of the 132 records that were indicated as "no" for PF3, 104 files matched actual BMI client data provided in the CTF. Differences in the figures for these two variables may be due to confusion about BMI cut-offs indicated in the IPRIT guide. The old IPRIT had different cut-off values for this risk factor compared to the new form. Each site of the POP has been designated to use the new forms but the database provided reflects the old IPRIT. Therefore, there is uncertainty as to which cut-offs the individual POP sites used for this risk factor. PF4 - Rate Of Weight Gain. PF4, rate of weight gain, had a completion rate of 36.2%. The IPRIT guide defines specific values of appropriate weight gain within certain time periods of the pregnancy. Definitions of appropriate weight gain are also based on the client's BMI. The completion rate of this factor was quite low which may be due to 54 confusion about cut-off values provided in the IPRIT. Reasons for the confusion have been cited in the analysis of the previous variable. Cross-tabulating this variable with weight data provided on the CTF would require complex calculations. The CTF provides due date information which could be used to determine the number of weeks gestation when the client's weight was assessed. Weight gain information is recorded in the CTF at pre-pregnancy, program intake and at last visit before delivery. Using the four variables, initial weight, intake weight and weight at last visit before delivery and comparing it with this risk factor would provide the cross-tabulation. The results, however, would be incomplete because weight data is only recorded in the CTF at specific time intervals. PF5 - Inadequate Nutrition PF5, inadequate nutrition, had the highest completion rate (62%) of this section and is based on the initial 24-hour diet recall recorded in each client's chart. The criteria used to assess nutrition are those of the BC Food Guide for Pregnancy and, where required, the registered dietitian-nutritionist at each site may assess specific situations not applicable to the criteria of the IPRIT provided. Cross-tabulating this risk factor with data of the CTF would provide incomplete information. Within the CTF twenty-four hour diet recall information is recorded at program intake and last visit before delivery. In the client's chart, however, more diet recalls are recorded and used as a basis for assessing this risk factor. Therefore, the CTF information is not complete enough to compare with IPRIT data for this risk factor. PF6 - Previous Child With Anomaly Or Disorder. PF6, previous child with anomaly, had a 22.9% completion rate. This may be due to the depth of the criteria 55 provided in the IPRIT guide. The current definition provided for this risk factor leaves the onus on the program staff member to question the client in depth about their previous children. PF7- Previous High Risk Infant. PF7, previous high risk infant, had a completion rate of 23.9% which was similar to the completion rate of PF6. Reasons for the low completion rate for this risk factor may be due to those previously cited for PF6. The incidence of risk factors PF6 and PF7 are rare and this may account for the low completion rates indicated for them. It may be speculated that much of the incomplete data for these risk factors should have been indicated as "no." PF8 - Multiple Pregnancy. PF8, multiple pregnancy, had a completion rate of 22.3%. Multiple pregnancy is defined as those clients expecting two or more infants. Multiple births represent 2% of all births and 16% of the low birth weight population (The Institute of Child Health, 1993). Based on the figures within the database, the proportion of POP clients with multiple pregnancy was 1.2% that suggests that this risk factor may be under reported. PF9- Birth Interval. PF9, birth interval, had a completion rate of 32.1 %. Timing between pregnancies is the basis for PF9 with the criterion being less than 2 years between pregnancies. The completion rate was 32.1% which may be due to the confusion about the basis of the two year time frame between births. PFlO- Grand Multipara. PFlO, grand multipara, had a completion rate of26.2%. This risk factor refers to whether a client has had 5 or more pregnancies. PF 10 was cross-tabulated with past pregnancy data recorded in the CTF. The following table shows the results of this analysis. 56 Table 3.4 Grand Multipara And Gravida PF1 0: Grand multipara (recorded in the IPRIT} Gravida (recorded in the CTF} Yes No lncompletes Total 47 2 5 or more pregnancies Less than 5 pregnancies 22 216 1 Don't knows 0 lncompletes 0 Total 70 219 15 743 0 55 813 64 981 1 56 1102 As the above table illustrates there were discrepancies between the crossreferenced variables. Of the seventy files which had PF10 indicated, there were twentythree client records that did not correspond to actual gravida information provided in the CTF. Conversely, ofthe 219 client files which had PF10 indicated as a "no," 216 records matched with actual gravida information provided in the CTF. The source of discrepancy is difficult to ascertain but it is of a sufficient level that causes concern about the quality ofthe data. PFll - Established Genetic Risk. PFll , established genetic risk, had a completion rate of23 .3%. This may be due to confusion about conditions that do and do not have clear genetic links between generations. PF12- Age 17 And Under/Age 36 And Over At Time of Delivery. The last risk factor ofthis section, PF12, had a completion rate of39.0%. This variable is based on the age of the client at delivery. This variable was cross-tabulated with actual client ages recorded in the CTF. The following table shows the results of this analysis. 57 Table 3.5 Age 17 And Under/Age 36 And Over At Time Of Delivery And Client's Actual Age PF12 : Age 17 and under/age 36 and over at time of delivery (recorded in the IPRIT} Client's age (recorded in the CTF} Yes No lncompletes Age 17 or less and 35 or more Age over 17 and less than 35 Inadmissible lncompletes Total 132 0 114 181 652 2 6 0 1 0 248 182 12 Total 144 947 3 8 2 672 1102 The data provided for these two variables showed apparent discrepancies . Of the 248 client files that had PF12 indicated, 132 matched actual client ages indicated in the CTF. Conversely, of the 182 records that indicated "no" for PF12, 181 matched the information provided for client ages in the CTF. The discrepancy in the data may be due to time frames selected for recording the data. Risk factor information pertains to age at delivery whereas CTF information may or may not be the client's age when she gave birth (for example, it may be the age at program intake). IPRIT- Section Two: Substance Abuse Risk Factors Table 3.6 presents data from the substance abuse section of the IPRIT. Substance abuse risk factors include exposure to second-hand smoke as well as any cigarette, alcohol or drug use. 58 Table 3.6 IPRIT- Substance Abuse Factors No Number of complete records Risk Description factor Yes SAF1 Smoking SAF2 Alcohol use SAF3 Inappropriate use of over the counter and treatment drugs SAF4 Other drug use 560 118 337 148 20 227 140 198 Completion rate Total 678 485 247 61 .5% 44 .0% 22.4% 1102 1102 1102 338 30 .7% 1102 Completion rates for this section ranged from 22.4% to 61.5 % which may reflect the difficulties in collecting information in this highly sensitive area. The following discusses each substance abuse risk factor in greater detail with respect to data quality. SAFl - Smoking. SAFl, smoking, had a completion rate of 61.5%. This variable was cross-tabulated with the variable, number of cigarettes smoked pre-pregnancy located in the CTF. The following table shows the results of this analysis. Table 3.7 Smoking And Number Of Cigarettes Smoked Pre-pregnancy SAF1 : Smoking (recorded in the IPRIT) Prepregnancy smoking (recorded in the CTF) Yes No lncompletes Total At least one cigarette per day No cigarette smoking Don't knows Inadmissible In completes Total 526 26 23 90 2 0 1 0 2 8 560 118 704 352 5 1 40 1102 152 239 3 0 30 424 The data provided for SAFl and pre-pregnancy smoking were inconsistent. Of the 560 client records which has SAFl indicated, 526 records matched information provided 59 about cigarette use in the CTF. Of the 118 records which had SAF1 indicated as "no", 90 files corresponded. The discrepancy may be due to the fact that SAF1 criteria also included exposure to second-hand smoke. However, if this was the case it would be expected that there would be more clients indicated for SAF1, however, there were fewer than the number of clients which had prepregnancy cigarette use indicated in the CTF. SAF2 - Alcohol Use. SAF2, alcohol use, had a completion rate of 44.0%. This variable was cross-tabulated with data from the CTF for pre-pregnancy alcohol intake. The following table shows the results of this analysis. Table 3.8 Alcohol Use And Number Of Drinks Consumed Pre-pregnancy SAF2 : Alcohol use (recorded in the IPRIT) Prepregnancy drinking (recorded in the CTF) Yes No lncompletes At least one drink per week No drinks Don't knows lncompletes Total 243 43 63 101 4 0 27 4 337 148 182 383 51 617 Total 468 547 5 82 1102 There were apparent discrepancies in the data for alcohol use between the IPRIT and the CTF as indicated by the above table. Of the 337 client records which had SAF2 indicated, 243 corresponded with prepregnancy drinking information provided in the CTF. Conversely, of the 148 records with "no" indicated for SAF2, 101 client files corresponded with CTF information. The discrepancy may be partially due to the time frame which client alcohol intake information was derived from. For example, alcohol use information from the CTF that was used as a basis for risk scoring could have been 60 taken from pre-pregnancy or program intake time periods. In addition, there may be confusion about whether the risk factor should be indicated if the T-ACE score measurement reveals potential risk of drinking. SAF3 - Inappropriate Use Of Over The Counter And Treatment Drugs. SAF3, inappropriate use of over the counter and treatment drugs, had a completion rate of 22.4%. This risk factor refers also to herb use in pregnancy. The low completion rate for this risk factor may reflect the difficulties a staff member must encounter in assessing this variable. Investigations of what constitutes safe and unsafe herbs in pregnancy remain to be incomplete which can lead to discrepancies in how data is collected for this variable. The current IPRIT guide also does not include inhalant use that may also contribute to the accuracy of data collected for this risk factor. SAF4- Other Drug Use. SAF4, other drug use, had a completion rate of30.7%. This variable combined with SAF3 was cross-tabulated with pre-pregnancy drug use information located in section six of the CTF. The following table shows the results of the analysis. 61 Table 3.9 Other Drug Use, Inappropriate Use Of Over The Counter And Treatment Drugs And Pre-pregnancy Drug Use SAF3: Inappropriate use of over the counter and treatment drugs and SA4 : Other drug use (recorded in the IPRIT) Prepregnancy drug use (recorded in the CTF) Yes No lncompletes Total Drug use at least once per week No drug use Don't knows lncompletes Total 122 24 23 165 1 0 7 6 153 195 248 764 2 88 1102 102 576 75 754 There were apparent discrepancies between these variables as illustrated by results in this table. Of the 140 to 160 client records which had either SAF3 or SAF4 indicated, there were 122 files that matched pre-pregnancy drug use information in the CTF. Conversely, 165 files specifying drug use within the CTF corresponded to negative responses to the risk factors SAF3 or SAF4. Discrepancies may be due to unclear definitions of what constitutes drug use. The number of records indicated for the risk factor SAF4 was approximately double the number of records indicating pre-pregnancy drug use. IPRIT- Section Three: Socio-Economic Risk Factors Table 3.10 presents data of the socio-economic section of the IPRIT. This section of the IPRIT addresses issues of the maternal social environment that are relevant to outcome of pregnancy. 62 Table 3.10 IPRIT- Socio-Economic Factors Risk Description factor Yes No Number of complete records SEF1 Single parenthood 518 88 SEF2 Delayed access to prenatal care 104 194 SEF3 Refusal/resistance to appropriate 45 225 services SEF4 Isolation -ethnic, language and/or 237 161 social SEF5 Limited learning ability/illiterate 92 202 342 147 SEF6 Marital problems, unstable relationship 254 148 SEF7 Inadequate housing SEF8 Financial problems 917 29 Completion Total rate 606 298 270 55.0% 27.0% 24.5% 1102 1102 1102 398 36.1% 1102 294 489 26 .7% 44.4% 1102 1102 402 946 36 .5% 85.8% 1102 1102 Completion rates ranged from 24.5% for SEF3 to 85.8% for SEF8. The following discusses each socio-economic risk factor in greater detail as it relates to issues of data quality. SEFl - Single Parenthood. SEFl , single parenthood, had a 55.0% completion rate. The IPRIT guide criteria indicates that this risk factor is marked positive for clients who are not married. SEFl was cross-tabulated with marital status information provided in the CTF. The following table shows the results of the analysis. 63 Table 3.11 Single Parenthood And Marital Status SEF1 : Single parenthood (recorded in the IPRIT) Marital status (recorded in the CTF) Yes No lncompletes Total Single Married, commonlaw or in relationship Inadmissible Total 355 4 160 84 36 455 395 699 3 0 518 88 5 496 8 1102 There were discrepancies between the figures derived from the IPRIT and CTF with respect to marital status. Of the 518 client files which has SEFl indicated, 355 records matched marital status information provided from the CTF. Eighty-eight records had "no" indicated for SEFl and of these 84 files matched marital status information provided in the CTF. SEF2- Delayed Access To Prenatal Care. SEF2, delayed access to prenatal care, had a completion rate of 27 .0%. This may be due to confusion about how to assess this variable. The criteria in the IPRIT guide lists some factors to consider such as no medical care by 20 weeks and no attendance at prenatal classes in a primipara. To ensure consistency in data collection for this risk factor, more precise criteria may be required. SEF3 - Refusal/Resistance To Appropriate Services. SEF3, refusal/resistance to appropriate services had a completion rate of 24.% which was the lowest of this section. This may reflect the difficulty in ascertaining whether this risk factor is present or not for a client. The criteria suggests refusal to services such as the Ministry of Human 64 Resources poses risk but does not provide greater detail than this. The onus remains with the program staff member working with the client to determine if the risk factor exists. SEF4 - Isolation - Ethnic, Language And/Or Social. SEF4, isolation, had a completion rate of 36.1 % which may reflect difficulties on behalf of the program staff member in assessing this risk factor. For example, a definition of the term "support" for SEF4 would be helpful to include in the criteria of this risk factor. SEF5 - Limited Learning Ability/Illiterate. SEF5, limited learning ability/illiterate had a low completion rate (26.7%). As is the case with SEF4, this may be due to how the risk factor is defined and reflect the difficulties in assessing this. For example, definitions of "illiterate" and "limited learning ability" for SEF5 would help to ensure standardization of data collection for this particular risk factor. SEF6 - Marital Problems Or Unstable Relationship Or Family Violence. SEF6, marital problems or unstable relationship, had a completion rate of 44.4% which may reflect the difficulties in assessing this variable. Because of the sensitive nature of this variable many clients may not disclose this information. SEF7 - Inadequate Housing. SEF7, inadequate housing, had a completion rate of 36.5%. This variable would be difficult to assess unless the program staff member does a client home visit. The proportion of home visits for 1994/95 was 33.9%; thus accurate data could be completed for this proportion of clients only. SEF8 -Financial Problems. The last variable of this section, financial problems, had the highest completion rate of this section (85 .8%). The following table shows the crosstabulation of this risk factor with client financial status information from the CTF. 65 Table 3.12 Financial Problems And Client's Financial Status SEF8 : Financial problems (recorded in the IPRIT) Financial status (recorded in the CTF) Yes No lncompletes Receiving income assistance 886 16 or low/inadequate income Not low income 16 13 15 0 Inadmissible 917 29 Total Total 102 1004 45 9 156 74 24 1102 As the above table indicates there were discrepancies between the two crossreferenced variables. Of the 917 records with SEF8 indicated, 886 corresponded to financial status information provided in the CTF. Of the 29 records that indicated "no" for SEF8, 13 records corresponded to data in the CTF section. Discrepancies in the data may be due to different income cut-offs being used. For example, some program sites may be using the low income cut-offs which are from 1985 and provided in the POP handbook while other sites may be using more updated cut-off values. IPRIT- Section Four: Emotional Risk Factors Table 3.13 presents data from the emotional risk factor section of the IPRIT. Emotional risk factors include the client's family history, mental health and self-esteem issues, as well as feelings the client has about their pregnancy. 66 Table 3.13 IPRIT - Emotional Factors Risk factor Description Yes No Number of Completion Total complete rate records EF1 EF2 EF3 EF4 Family history of abuse/neglect Mental health problems Low self-esteem Inability to cope/anxiety regarding pregnancy and baby Unrealistic expectations Unwanted ~r 61 141 90 204 240 177 184 189 202 294 417 373 18.3% 26 .7% 37.8% 33.8% 1102 1102 1102 1102 116 213 153 191 329 344 29.9% 31 .2% 1102 1102 EF5 EF6 Completion rates for this section ranged from 18.3% for EF1 to 37.8% for EF3. The following provides more detail about each emotional risk factor with respect to data quality issues. EF1 -Family History Of Abuse/Neglect. EF1, family history of abuse/neglect, had the lowest completion rate of the entire IPRIT section (18.3%). This may reflect the difficulties that the program staff member encounters when trying to assess this variable. EF2 - Mental Health Problems. EF2, mental health problems, had a completion rate of 26. 7%. This may reflect the difficulties in assessing these variables. Although the criteria in the IPRIT guide is extensive it may be excluding important information such as a history of post partum depression. Information provided for this risk criterion duplicates criteria outlined for other risk factors such as SEF4, EF1 and EF4. EF3 - Low Self-esteem. EF3, low self-esteem, had a completion rate of 37.8%. Program staff may find this risk factor difficult to assess given the current criteria provided. For example, a definition of self-esteem for the risk factor EF3 would provide for improved data accuracy. 67 EF4- Inability To Cope/Anxiety Regarding Pregnancy And Baby. EF4, inability to cope/anxiety regarding pregnancy and baby, had a completion rate of 33 .8%. This may reflect the difficulty in assessing this variable as it involves subjective judgement on the part of the program staff member. EF5 - Unrealistic Expectations. EF5 , unrealistic expectations, had a completion rate of 29.9%. This low rate may be due to the fact that assessment of this factor involves subjective judgement on the part of the program staff member who works with the client. EF6 - Unwanted Pregnancy. EF6, unwanted pregnancy, had a completion rate of 31.2%. This may reflect the difficulty in assessing this variable as it involves subjective judgement. It is difficult to ascertain whether this risk factor should be indicated or not for the clients who initially do not want the pregnancy and later decide to carry on with it and demonstrate healthy mother/infant bonding. Summary The IPRIT form includes a broad range of risk factors . The major data quality issues highlighted in this chapter included repetition in risk factor definitions, incomplete criteria provided in the IPRIT guide and discrepancies with corresponding information in the CTF. There was a wide variation on item-per-item completion rates of the risk factors . The highest rate of completion was for the risk factor associated with financial problems. The large proportion of missing data for the various risk factors of the IPRIT can be attributed to the fact that data entry for this section has not been standardized. It can be speculated that those responsible for data entry left risk factors incomplete for reasons that include 68 uncertainty as to whether a specific risk factor existed or when a risk factor was absent. Every field of the computerized IPRIT should have a "yes" or "no" answer. If the computerized version of the IPRIT was programmed for error control there would have been less likelihood of missing data. Error control refers to the close control of the accuracy of the data at the input level, in storage, and input (Duncan, 1981 ). On the IPRIT form there is a space allotted for the program staff member to enter comments, however, this is not available in the computerized version. To help verify the accuracy of the IPRIT data, it would be beneficial to include sections for comments on the IPRIT computerized form. This chapter presented and discussed the data from the IPRIT section of the 1994/95 database and a number of key issues were discovered in relation to data quality. The following chapter presents data from the CTF section ofthe 1994/95 database. 69 Chapter 7 RESULTS - CLIENT TRACKING FORMS This chapter evaluates data that was recorded in the Client Tracking Forms (CTF) of the 1994/95 POP database. The CTF is subdivided into eleven sections: program information, referral source to program, intake, client characteristics, past pregnancy data, client monitoring, project contact, referrals from program, project outcomes, alcohol and smoking data. Data recorded in the CTF is shown in the following tables. Comments are made with respect to the completion rates, to the accuracy and validity of the recording as well as possible difficulties encountered by staff in completing the form . Completion rates were calculated based on the number of completes including specific "don't know" responses divided by the total number of client records who were indicated to have stayed in the program until completion. Variables in which the total numbers were not 1102 records are indicated with an asterisk and explanations provided following the table. Qualitative data collected on the form are presented in separate tables and where appropriate the information was categorized. Client Tracking Form- Section One: Program Information Table 4 presents the data from the program information section of the CTF. This section of the CTF is intended to assign a numerical identification for each client referred to the POP, a code that defines the client that started the program. The variable project location, is intended to identify the client with a particular POP site. Client provincial 70 medical service plan numbers are collected in order that clients can be tracked in their access to other government services. Table 4 CTF - Section One : Program Information Variable Completes Client ID numbers Client began program 1100 1102 [yes=1 098, no=4] Project location 1102 Client care card number 335 lncompletes Don't Completion knows rate 2 0 606 161 Total 99.8% 100.0% 1102 1102 100.0% 30.4% 1102 1102 The data shows inconsistencies in the assignment of client numbers. Some code numbers were two letters followed by two digits (e.g. SD66) whereas others included combinations of digits and characters. Some sites had entered only two digits to represent the year in which the client's expected due date (EDD) occurs which created difficulty in interpreting which fiscal year this should be reported in. For example, if 94 was entered as part of the ID number, this could mean that the client's due date occurred either in 1993/94 or 1994/95 . Within the ID number, the year in which the client's EDD is usually incorporated. For example, a code number may read as 949501 with the first four digits being the fiscal year (between April 1, 1994 to March 31, 1995) in which that client is expected to deliver her baby. The last variable of this section, client provincial medical service plan numbers, had only a 30.4% completion rate. This is an important variable for this database since it is the focus of tracking identification for the client in her contacts with the system. This 71 variable may be poorly recorded due to a perception that collection of this information may interfere with the client's right to privacy. Client Tracking Form- Section Two: Referral Data Table 5 presents the data relevant to the source of the referral made for a client to the POP. Table 5 CTF - Section Two: Referral Data Variable Source of referral Referral date Weeks gestation at referral Completes 1102 1033 1102 lncompletes Inadmissible Completion Total rate 61 8 100.0% 93.7% 100.0% 1102 1102 1102 The completion rates for this section were 100%, 93.7% and 100% for source of referral, referral date and weeks gestation at referral respectively. The variable, referral date, had 8 inadmissible records and 61 incomplete records. Inadmissible records included coding errors in dates. Client Tracking Form- Section Three: Intake Data Table 6 presents data from the CTF that is related to the intake of the client to the POP. 72 Table 6 CTF - Section Three: Intake Data Variable Intake assessment date Due date Did client begin program? If No, "why not?" If "other", give reason why the client did not begin program? Completes Inadmissible 1096 4 1096 1102 [yes=11 02, no=636] 290 [not at risk=11, refused/not interested=1 07, other=172] 13 [No pregnancy/Client died=2, Miscarriage/Abortion=11] 6 Don't knows 2 Completion Total rate 99 .6% 1102 99 .5% 100.0% 1102 1738 45.6% 636* 7.6% 172** * represents number of clients that did not begin the program **represents number of clients who did not begin the program and had reasons cited as "other" The first variable of this section, intake assessment date, had a completion rate of 99.6% with four records which had inadmissible dates. The variable, client' s due date, also had a 99.5% completion rate with only six inadmissible dates being indicated. The variable, "did client begin program?", had 100% completion. There was a discrepancy in the reasons recorded for not beginning the program. Eleven did not begin because they were not high risk, 107 did not begin because they were not interested and 172 did not begin for reasons cited as "other" . Only thirteen records, however, had specific reasons cited for not beginning the program. Client Tracking Forms- Section Four: Client Characteristics Table 7.1 presents data related to demographic information about POP clients. 73 Table 7.1 CTF - Section Four: Client Characteristics Variable Completes Age Marital status 1093 1094 [married=191, common law =34 7, single=395, relationship =161) First 1094 language [english=1 045, other (specify) =49] Don't Completion Total knows rate 8 9 100.0% 100.0% 1102 1102 8 100.0% 1102 As shown in Table 7.1, the completion rates for this section were alll 00%. Table 7.2 presents data related to specific languages spoken by the client. Data in this table summarized the other languages that were indicated in the previous question. Table 7.2 Other language Variable African Asian East Indian European Middle East First Nations lncompletes Total Total 4 9 8 22 2 2 2 49 As shown in Table 7.2, only two records were incomplete. Table 7.3 presents additional demographic data about the POP clientele. 74 Table 7.3 CTF- Section Four: Client Characteristics Variable Completes Ethnic background 1098 [caucasion=752, native lndian=321, indocanadian=4, chinese =3, vietnamese=4 , latin american=6, other=8) 325 If ethnic background is Native Indian, state client's status eg. Metis Education Employment status 1074 1093 lncompletes Inadmissible Don't Completion Total knows rate 4 2 25 9 3 100.0% 1102 101.2% 321* 97 .5% 99 .2% 1102 1102 *total represents number of responses that indicated Native Status The completion rates for this section included 100.0%, 101.2%, 97.5% and 99.2% for ethnic background, native status, education background and employment status respectively. There are only minor discrepancies in the number of records completed in this section. For example, the first question of this section asks about the client's ethnic background of the number of responses indicating native indian was 321. The following question asks if the ethnic background is native indian please indicate what the client's status is. For the latter question there were four more answers than the responses indicating that the client was native status. Why this difference occurred is unknown. Table 7.4 presents data related to the occupation of clients who have indicated that they were employed. 75 Table 7.4 CTF - Section Four: Client Occupation Information Total Variable Factory worker/Deckhand Para-professional/Professional Works in service industry Inadmissible Total 7 17 90 17 131 For Table 7.4, related occupations were categorized together. Inadmissible information included answers which were numbers, answers that indicated the client was on medical leave, was a student, or specified only part-time or volunteer status and probably should have been classified as not employed. Table 7.5 presents information about the client's financial situation. In addition, this section ofthe CTF included the individual client's T-ACE scores. Table 7.5 CTF- Section Four: Financial Information And T-ACE Scores Variable Completes Financial situation 1078 [income assistance=702, low/inadequate income=302 , not low income=74] 1054 [income assistance=31 0, low/inadequate income=240, not low income=82 , N/A=422] 1063 Partner's financial situation T-ACE Score lncompletes Inadmissible Completion Total rate 22 2 97.9% 1102 47 95.7% 1102 39 96 .5% 1102 Completion rates for client financial situation and partner financial situation were 97.9% and 95 .7% respectively. Reasons for this may include the lack of clear criteria for low income cut-offvalues. r---------------------------------- -- --76 Client Tracking Forms - Section Five: Past Pregnancy Data Table 8 presents data about the obstetrical history of the client. Table 8 CTF - Section Five: Past Pregnancy Data Variable Number of pregnancies Number of deliveries Number of term deliveries Number of elective abortions Number of spontaneous abortions Number of living children Number of stillbirths Number of low birth weight Attended prenatal classes during a previous pregnancy Has client ever previously been a POP client Completes lncompletes Inadmissible Don't know Completion Total rate 1046 708 716 685 56 394 386 416 94 .9% 64.2% 65.0% 62.2% 1102 1102 1102 1102 692 410 62.8% 1102 710 665 671 1087 392 437 431 12 3 64.4% 60 .3% 60 .9% 98 .6% 1102 1102 1102 1102 1089 10 3 98 .8% 1102 Completion rates for this section ranged from 60.3% for number of stillbirths to 98.8% for the question asking if the client has ever previously been a POP client Missing data for this section can likely be attributed to the fact that where a figure of zero or not applicable should be indicated for past pregnancy data, it was left blank. In addition, the lack of definitions for terms such as spontaneous abortion, stillbirth and term delivery may contribute to the high number of incomplete records for these questions. The lack of specific definitions means that data may be inconsistently recorded since definitions vary depending on the source used. For example, Health and Welfare Canada defines spontaneous abortions as the complete expulsion or extraction from its mother of a fetus or embryo weighing less than 500 g irrespective of gestational age. The World 77 Health Organization (1994) defines abortion as the expulsion of products of conception up to 28 weeks gestation. Others describe spontaneous abortions as the complete expulsion or extraction from its mother of a fetus or embryo at twenty weeks gestation or less. The question "Attended prenatal classes during a prevwus pregnancy?" was completely answered for 98.5% of the records. The question does not, however, specify the type of prenatal classes attended or the regularity of attendance. For example, prenatal classes are typically subdivided into two groups with the first set of classes being early bird classes which are typically attended in the first trimester of pregnancy and labour and delivery classes attended in the third trimester. This question does not specify if previous attendance in prenatal classes includes attending all class types and/or implies full attendance. For clients who are pregnant for the first time, the question "has client previously been a POP client?" does not apply to them. It is also unclear as to whether prior participation in related programs such as those funded by the Canada Prenatal Nutrition Project (outreach programs for high risk pregnant women that are funded by the federal government) should be included or excluded which may account for the ten records which were incomplete. Client Tracking Forms- Section Six: Client Monitoring Table 9.1 presents data about the client's last assessment with the program. 78 Table 9.1 CTF - Section Six: Date Of Assessment Variable Completes lncompletes Coding Don't Completion Total errors knows rate Date of assessment (last visit before deliver 1068 96 .9% 38 1102 As Table 9.1 shows 96.9% of the records had assessment dates indicated for the client's last visit before they delivered their baby. Table 9.2 presents data relevant to the monitoring of the weight of the client. Table 9.2 CTF - Section Six: Client Weight Monitoring Variable Date of assessment (last visit before delivery) Pre-pregnancy weight Program intake weight Weight (last visit before delivery) Body Mass Index (BMI) - pre-pregnancy Completes lncompletes 1068 38 441 446 450 652 654 643 670 432 Coding Don't errors knows 4 9 2 5 Completion rate Total 96.9% 1102 40.8% 40 .7% 41 .3% 1102 1102 1102 60 .8% 1102 The variables of this section had low completion rates. The pre-pregnancy weight was recorded in 40.8% of the records . This might be attributed to the fact that many clients did not remember what their weight was before they became pregnant. Nevertheless, pre-pregnancy weight is used as the basis for calculating the body mass index (BMI) which appears as the last variable in this section. Two-hundred and twentynine additional records to those files which had pre-pregnancy weight completed had a 79 BMI indicated which suggests that the records have failed to assess the pre-pregnancy weight that was in fact available. The BMI is an assessment of body weight appropriateness based on a weight to height ratio calculation. It is noted, however, that height information is not collected on the CTF and there is no way of verifying the accuracy of the data calculated for the BMI. It includes Table 9.3 presents client monitoring data related to food intake. information about meals and snacks consumed. Table 9.3 CTF - Section Six: Food Intake (number of servings per day based on 24 hour recall) Variable Completes Number of meals/day at intake (meal includes 3-4 food groups) Number of meals/day at last visit before delivery Number of snacks/day at intake (1-2 food groups) Number of snacks/day at last visit lncompletes 948 153 839 240 819 739 Don't knows Completion Total rate 86.1% 1102 23 78.2% 1102 277 6 74.9% 1102 341 22 69.1% 1102 Information collected about meals and snacks is based on food groups consumed in accordance with the BC Food Guide for Pregnancy. Completion rates varied from 69.1 % for number of snacks/day at last visit to 86.1% for number of meals at intake which is inconsistent with the variable indicating that 96.9% of client records had assessments completed. Table 9.4 provides client monitoring data related to the individual food groups of the BC Food Guide for Pregnancy that were consumed by clients based on their food intake in a twenty-four hour time period. 80 Table 9.4 CTF - Section Six: Food Intake (Number of servings of each food group) Variable Number of grain products/day at program intake Number of grain products/day at last visit before delivery Number of vegetables and fruits/day at program intake Number of vegetables and fruits/day at last visit before delivery Number of milk and milk products/day at program intake Number of milk and milk products/day at last visit before delivery Number of meat and alternatives/day at program intake Number of meat and alternatives/day at last visit before delivery Completes lncompletes Don't knows Completion Total rate 1030 57 15 94.8% 1102 937 139 26 87.4% 1102 1001 88 13 92 .0% 1102 922 156 24 85.8% 1102 908 181 13 83 .6% 1102 881 197 24 82.1% 1102 993 96 13 91 .3% 1102 931 146 25 86.8% 1102 Dietary intake is based on a recollection of foods consumed by the client within the last twenty-four hours. Completion rates for this section varied from 82 .1% for milk and milk products consumed per day at last visit to 94.8% for grain products consumed at program intake. Although the completion rates were relatively high for this section, they were inconsistent with the completion rates provided for intake of snacks and meals in the previous table and inconsistent with the number of records which were indicated to have assessments completed (n= 1068). Table 9.5 presents client monitoring data related to beverages consumed. The information includes intake ofbeverages containing caffeine, sweetened drinks and water consumed in a twenty-four hour time period. 81 Table 9.5 CTF - Section Six: Food Intake (caffeine, sweetened drinks and water) Variable Number of coffee (percolated , drip, caffeinated)/day at program intake Number of coffee (percolated, drip, caffeinated)/day at last visit before delivery Number of coffee (instant, caffeinated)/day at intake Number of coffee (instant, caffeinated)/day at last visit before delivery Number of tea servings (caffeinated)/day at program intake Number of tea servings (caffeinated)/day at last visit before delivery Number of colas (caffeinated)/day at program intake Number of colas (caffeinated)/day at last visit before delivery Number of other fluids: pops and sweetened fruit drinks/day at program intake Number of other fluids : pops and sweetened fruit drinks/day at last visit before delivery Number of water servings/day at program intake Number of water servings/day at last visit before delivery Completes lncompletes Don't knows Completion rate Total 303 790 9 28.3% 1102 211 867 24 21 .3% 1102 35 1059 8 3.9% 1102 43 1039 20 5.7% 1102 287 806 9 26 .9% 1102 192 890 20 19.2% 1102 250 844 8 23.4% 1102 137 944 21 14.3% 1102 388 705 9 36 .0% 1102 244 838 20 24 .0% 1102 595 487 20 55 .8% 1102 496 568 38 48.5% 1102 Completion rates for this section varied from 3.9% for number of cups of coffee (instant, caffeinated) consumed at program intake to 55 .8% for number of water servings at program intake. The low completion rates for this section could be attributed to the fact that where a figure of zero should have been indicated, the field was left blank. While it is not clear why there is variation in the response rates for the individual items, 82 the overall low completion rate does not allow for any data analysis with respect to the intake of these food items. Table 9.6 shows the data regarding intake of iron and folate. Criteria used to define good and other sources of iron and folate are located in Appendix E. Table 9.6 CTF - Section Six: Food Intake (Iron and Folate Sources) Variable Iron rich foods/day at program intake Iron rich foods/day at last visit before delivery Excellent sources of iron/day at program intake Excellent sources of iron/day at last visit before delivery Other sources of iron/day at program intake Other sources of iron/day at last visit before delivery Folate rich foods/day at program intake Folate rich foods/day at last visit before delivery Excellent sources of folate/day at program intake Excellent sources of folate/day at last visit before delivery Other sources of folate/day at program intake Other sources of folate/day at last visit before delivery Completes lncompletes 264 824 Don't knows 14 Completion rate 25.2% Total 241 842 19 23 .6% 1102 382 708 12 35.8% 1102 433 646 23 41.4% 1102 759 331 12 70.0% 1102 717 362 23 67.2% 1102 252 835 15 24 .2% 1102 231 852 19 22 .7% 1102 536 554 12 49.7% 1102 550 529 23 52 .0% 1102 721 369 12 66 .5% 1102 700 378 24 65 .7% 1102 1102 Completion rates for this section varied from 22.7% for folate rich foods consumed at last visit before delivery to 70.0% for other food sources of iron consumed at program intake. 83 Table 9.7 shows the information collected about cigarettes smoked at time intervals that included pre-pregnancy, program intake and last visit before delivery. Table 9.7 CTF - Section Six: Cigarette Smoking Variable Number of cigarettes smoked/day (pre-pregnancy) Number of cigarettes smoked/day (program intake) Number of cigarettes smoked/day (last visit before delivery) Completes lncompletes Don't knows Completion Total rate 710 387 5 64 .9% 1102 502 597 3 45.8% 1102 374 714 14 35 .2% 1102 Completion rates varied from 35.2% for number of cigarettes smoked per day at last visit before delivery to 64.9% for number of cigarettes smoked per day at program intake. Completion rates declined at each stage the data was collected during pregnancy. In addition, the completion rates did not correspond to the number of records indicating that the assessment was completed for this section of the CTF. This may be partly due to the fact that where a figure of zero should have been indicated, it was left blank. It is not clear whether the decline in the completion rate is due to a real reduction in the incidence and amount of cigarette smoking. Table 9.8 presents data related to alcohol use and the intent of this section is to assess the program's impact on being able to reduce the use of alcohol. 84 Table 9.8 CTF - Section Six: Alcohol Use Variable Number of drinks/week (prepregnancy) Number of drinks/week (program intake) Number of drinks/week (last visit before delivery) Completes In completes Don't Completion Total knows rate 474 628 0 43.0% 1102 71 1028 3 6.7% 1102 30 1064 8 3.4% 1102 As was the case with cigarette smoking information, completion rates declined at each stage in the pregnancy the data was collected. Completion rates decline from 43.0% pre-pregnancy to 3.4% at the last visit. If, in fact, the incompletes mean the client was not drinking anything at the last visit before delivery, then the reduction in drinking is dramatic but the analyst can have little confidence in interpreting a blank field as a zero. Table 9.9 provides data about illicit drug use at each stage of pregnancy. Table 9.9 CTF - Section Six: Illicit Drug Use Variable Number of drugs used/week (pre-pregnancy) Number of drugs used/week (program intake) Number of drugs used/week (last visit before delivery) Completes lncompletes Don't knows Completion rate Total 78 1024 0 7.1% 1102 47 1052 3 4.5% 1102 275 819 8 25.7% 1102 Unlike the previous two variables for smoking and alcohol use, completion rates were variable with the highest rate being indicated for data collected at the last visit before delivery (25 .7%). The lowest rate of completion for this section was 4.5% for 85 number of drugs used per week at program intake. The low number of completed records for this section may be attributed to the fact that where a zero should have been indicated for no drug use, the field was left blank. The low rate of completion may also reflect the highly sensitive nature of this variable and the difficulties in collecting this information from the client. Table 9.10 present data about the types of drugs used. Table 9.10 CTF - Section Six: Type Of Drugs Used-Qualitative Data Variable Total Illicit drugs (acid, mushrooms, pot, hash, cocaine, LSD, oil, street drugs) 139 Illicit and over the counter drug combinations 2 Prescription and over the counter drugs (Adavan, Amoxicellin, Diclectin, 22 antibiotics, antinausea, mipramine, prozac, psychiatric meds, insulin, ventolin, clorazapam) 64 Alcohol Alcohol and hard drug combinations 42 Alcohol and prescription drugs or alcohol and over the counter drugs 2 Over the counter drugs 8 Quit drinking and drugs 1 year ago No response 15 Don't know 3 301 Total This qualitative variable generated responses that included over the counter and illicit drugs. In addition, it included information about alcohol use that is not relevant to this question. The high number of inadmissible responses for this question may be attributed to the fact that it is not clear whether the type of drug specified was to include illicit or non-illicit types. Client Tracking Forms- Section Seven: Project Contact Table 10.1 presents data relevant to evaluating intensity of service provision. 86 Table 10.1 CTF - Section Seven : Counselling Contacts Completes Program staff and visit type Home visits with health professional Site visits with health professional Phone visits with health professional Other visit types with health professional Home visits with outreach worker Site visits with outreach worker Phone visits with outreach worker Other visit type with outreach worker lncompletes Completion Total rate 404 210 102 384 796 698 892 1000 718 306 36.7% 19.1% 9.3% 34 .8% 72.2% 1102 1102 1102 1102 1102 673 326 122 275 429 776 980 827 61 .1% 29 .6% 11 .1% 25 .0% 1102 1102 1102 1102 Completion rates for this section varied from 9.3% to 72.2%. The variable completion rates for this section could be attributed to the fact that where a figure of zero should have been indicated, the field was left blank. Table 10.2 presents data about the utilization ofPOP services. Table 10.2 CTF- Section Seven : Other Contacts Variable Number of attendances at drop-ins (number of clients who attended drop-ins at least once) Number of clients who canceled at least one appointment and includes not home, no show, and other attempts made Receiving food supplements Completes lncompletes Completion Total rate 669 433 60.7% 1102 203 899 18.4% 1102 1087 [yes=998, no=89] 14 98 .6% 1102 87 The completion rates for this section ranged from 18.4% to 98 .6%. The variable completion rates for this section could be attributed to the fact that where a figure of zero should have been indicated, the field was left blank. The variable "receiving food supplements" had a completion rate of 98.6%. A survey of POP coordinators (Code, 1995) revealed that all sites provide food supplements although the type and amount varies across sites. Hence, it is difficult to assess the extent of supplementation provided and it is also difficult to ascertain whether the supplement was actually consumed by the client. Table 10.3 present data about physician contacts made by the client. Table 10.3 CTF- Section Seven: Physician Contact Information Variable Completes lncompletes 1088 Seeing a physician for prenatal care? [yes=1 080 , no=8] Date of first physician contact 835 14 267 Completion Total rate 98 .7% 1102 75.8% 1102 Completion rates were 98.7% for whether the client was seeking prenatal care from a physician and 75 .8% for date of first physician contact. The low completion rate for the last question may be attributed to the fact that the client may not recall the exact date they first saw their physician. Client Tracking Forms- Section Eight: Referrals Table 11 presents data about types of referrals made by POP staff. 88 Table 11 CTF - Section Eight: Referrals Agency which POP staff referred client to Mental Health Alcohol and Drug Programs Ministry of Social Services Health Unit Physician Nobody's Perfect Other (specify) lncompletes Completion Total During lncompletes Completion At program rate program rate discharge 28 63 1074 1039 2.5% 5.7% 14 17 1088 1085 1.3% 1.5% 1102 1102 182 920 16.5% 71 1031 6.4% 1102 496 177 102 606 925 1000 45.0% 16.1% 9.3% 288 66 253 814 1036 849 26.1% 6.0% 23 .0% 1102 1102 1102 354 748 32 .1% 233 869 21.1% 1102 Completion rates varied from 1.3% for referrals made to Mental Health at program discharge to 45% for referrals made to the Health Unit during the program. The low completion rates for this section is likely due to not recording a zero if the client was not referred to that particular agency. Client Tracking Forms- Section Nine: Program Outcome Table 12.1 presents data about birth outcomes. 89 Table 12.1 CTF- Section Nine: Program Outcome Variable Outcome of pregnancy Weeks gestation Infant birthdate Birthweight Medical complications Completes lncompletes Inadmissible Completion Total rate 1099 [single live birth=1 006, multiple live birth=18, stillbirth=6, miscarriage=53 , therapeutic abortion=16] 1064 2 99.8% 1102 38 96 .6% 1102 1034 68 93 .8% 1102 1014 429 88 673 92 .0% 38 .9% 1102 1102 Outcome, weeks gestation and birth data are virtually complete. For this section, there is no specification of where this information should be obtained from (i.e. medical records, etc). Therefore, the collection ofthis information may not standardized. Birth outcome information had a high completion rate, however, the three variables subsequent to it (weeks gestation, infant birthdate and birthweight) as well as the variable "did client stay in program?", did not correspond as would be anticipated. It is unclear whether the 673 records indicated as incomplete for medical complications represented no complications. As shown in Table 12.2, however, those with medical complications included 96 clients with no complications. Given this nonspecificity of the categories little can be concluded about outcome in this area. 90 Table 12.2 presents data about medical complications recorded by POP staff . Table 12.2 CTF- Section Nine: Medical Complications Information-Qualitative Data Variable Total Medical condition during pregnancy Baby outcome Delivery complications Pregnancy complications Pre-existing medical condition None Total 9 25 269 30 4 94 435 Table 12.3 presents data about visits that the client made to their physician as well as breastfeeding information. Table 12.3 CTF- Section Nine: Physician (MD) Visits Variable Did client attend MD visits this pregnancy? Breastfeeding at hospital discharge Breastfeeding at one month contact Completes lncompletes Inadmissible Completion rate Total 429 [yes=429 , no=O] 1028 [yes=820, no=159, don't know=49] 964 [yes=510, no=248 , don't know=206] 673 38 .9% 1102 74 93 .3% 1102 138 87.5% 1102 Completion rates varied from 38.8% for physician (MD) visits during pregnancy to 93 .3% for breastfeeding information collected at hospital discharge. The breastfeeding follow-up data showed discrepancies. The reporting fiscal year for this particular database was April 1, 1994 to March 31 , 1995 and the cut-off date for the individual sites of POP for data submission was April 30, 1995 . Therefore, for clients 91 who delivered their babies near the end of the fiscal year, the question about client breastfeeding status at one month contact may not be accurate. A category of don't know is provided but there were ten files which also had 9's (i.e. don't knows) indicated. For clients who supplement infant formula with breastfeeding, it is unclear whether a positive response for breastfeeding should be indicated. Specific program objectives include that the client should breastfeed for a minimum of six weeks, however, the measurement of continued breastfeeding is only for four weeks. If an answer of "no" to breastfeeding at hospital discharge was indicated, it also should be recorded as "no" for one month discharge, however, the variables did not correspond. Table 12.4 provides reasons why a client discontinued breastfeeding. Table 12.4 CTF - Section Nine: Reasons Why Breastfeeding Discontinued Variable Total No response/Not applicable Mother/infant separation (e.g. death, adoption, apprehension) Infant complications Concerns of adequate milk supply Advice from health professional to discontinue breastfeeding Body image issues/history of sexual abuse Started supplementing with infant formula Maternal complications (e.g. sore breasts, cracked nipples, medical complications) Substance abuse issues Mother's choice Client identified lack of support Difficulties with milk supply Mother taking medications Mother had mental challenges Client contact discontinued/breastfeeding status unknown Total 97 18 12 2 7 2 4 22 8 68 2 10 6 2 7 267 92 The number of responses that indicated reasons why breastfeeding was discontinued (n=267) was greater than the number of responses for the question that indicated breastfeeding had been discontinued in the previous (n=248). Table 12.5 presents data about the completion of the alcohol and smoking followup sheets. Any client who was in the program and had the risk factors of smoking or alcohol use, should have the respective follow-up information completed. Table 12.5 CTF- Section Nine: Alcohol Use And Smoking Follow-up Variable Yes No lncompletes Completion rate Alcohol use follow-up sheet filled out Smoking follow-up sheet filled out 462 348 544 230 292 328 73.5% 70 .2% Total 1102 1102 Completion rates for alcohol and smoking follow-up were 73.5% and 70.2% respectively. Referring to the IPRIT, 337 clients had a risk factor of alcohol use indicated, however, 462 clients had an alcohol use follow-up sheet completed. Clients who had a risk factor of smoking indicated was 560, however, only 544 of these clients had a smoking follow-up sheet completed. Client Tracking Forms - Section Ten: Alcohol Data Table 13.1 provides data about the coping methods used to avoid drinking. 93 Table 13.1 CTF- Section Ten: Coping Methods- Qualitative Data Variable Total Alcohol and drug treatment/programs or AA/Support Group Attends cultural activities Avoid situations (e.g . friends, family) Changed lifestyle due to family deaths related to alcohol Counselling/treatment centre Cries/blows up at husband or scream in pillow/at dad Drunk once per year/drank 1 week ago "weak one" Eat more Family support Feels better not drinking Feels drinking habits are fine or has no problem/desire Keep busy/sports activities/studying Nap/shower Not worth it/too expensive/look stupid Occasional/social drinking Quit completely/cold turkey or quit due to pregnancy Self healing/determination Takes little sips/limits intake or reduce or trying to abstain Travels Very easy not to drink Don't know Inadmissible Total 7 1 6 1 3 2 1 1 3 1 9 3 1 1 6 34 1 6 1 1 21 16 126 Total responses for this section did not correspond to the total of 462 positive responses indicated for completion of alcohol follow-up information. Inadmissible responses included answers that were numbers as well as responses indicating no coping method. Table 13.2 presents data about number of drinks a client consumes. 94 Table 13.2 CTF- Section Ten: Number of Drinks Variable 0-5 >5 and <20 >20 and <57 99 Total Total 83 196 52 12 331 Total responses did not correspond to the number of positive responses indicating that the alcohol follow-up sheet was completed. Table 13.3 presents data about the drinking patterns of the client and whether there is a past history of treatment for alcohol abuse. Table 13.3 CTF- Section Ten : Alcohol Consumption Variable Drinking patterns If daily, average number per day If binge, frequency of binging Past history of treatment Completes lncompletes Completion Total rate 296 45 168 417 100.0% 100.0% 464 462 236 379 [:tes = 68, no= 311] 226 83 100.0% 82 .0% 462 462 Completion rates ranged from 64.1 % for drinking patterns to 82.0% for past history of treatment. The number of total responses did not match the number of records which were indicated to have alcohol follow-up information completed (n=462). 95 Table 13.4 presents data about dates when the client sought former treatment for alcohol use. Table 13.4 CTF- Section Ten : Prior Treatment Times - Qualitative Data Variable 1987 to 1990 1991 to 1994 No response Total Total 7 28 35 70 The total number of responses indicating pnor treatment times (n=35) did not correspond to the total number of responses for past history of treatment indicated (n=68). Table 13 .5 presents data about treatment programs that the client had previsouly attended. Table 13.5 CTF- Section Ten : Prior Treatment Places Variable AA or AA and D&A ACOA or AI-Anon Abbotsford/Peadonville Alcohol Foundation Alcohol and Drug Counselling Counselling Alcohol and Drug Centre - Turning Point Alberta Calgary, Alberta- Sunrise Res Total Total 5 2 2 15 96 The total number of responses did not correspond to the number of responses for past history of treatment indicated (n=68) or prior treatment times (n=35). Table 13 .6 presents data about the client's personal goals with respect to alcohol use. Table 13.6 CTF- Section Ten : Personal Goals Variable Completes Personal goals 392 [no change=47, reduce =33, abstain=317] lncompletes Completion Total rate 70 84 .9% 462 The completion rate for this variable was 87.0% which does not correspond to the number records that were indicated to have alcohol follow-up information completed (n=462). Client Tracking Forms- Section Eleven: Smoking Data Table 14.1 presents data about the smoking habits of the clients as well as information about second-hand smoke exposure. 97 Table 14.1 CTF- Section Eleven: Smoking Information Variable Triggers for smoking Number of attempts at cessation Do other members of your household currently smoke Are you ever exposed to "second-hand" smoke Personal goal Completes lncompletes Inadmissible Completion rate 327 217 60 .1% 544 411 133 75.6% 544 522 22 96.0% 544 485 59 89 .2% 544 586 [no change=129 , reduce=203, guit=234] 0 107.7% 544 4 Total Completion rates varied from 60.1% for triggers for smoking to 96.0% for the question "do other members of your household currently smoke?". These did not correspond to the 544 records indicated to have had smoking follow-up information collected. The last variable of this section had more responses than the total number of records (n=544) indicated to have had smoking follow-up information collected. Table 14.2 provides qualitative information about methods the clients used to stop smoking currently and in the past. 98 Table 14.2 CTF- Section Ten : Methods of Cessation Methods of cessation Total Avoid/abstain or doesn't want to 4 Chew on pencils 1 Cold turkey 6 hours only Cold turkey/acupuncture 1 Cold turkey/cessation group or cold turkey/church support 3 Cold turkey/exercise or cold turkey/hypnotist 2 Cold turkey/marks number 1 Cold turkey/nicorettes/patch or cold turkey/gum 9 Cold turkey/sick or had flu 3 Cut down or weaning/reduce 25 Eating 1 Exercise and non-smoking tapes Gross Nicorettes No money 4 On her own 1 Patches or patch/cold turkey 6 Quit at conception but smoking now or give away pack 2 Quit completely/cold turkey 158 Quit due to pregnancy 14 Stay with non-smoker/stay away from smokers or support of 3 family Taper down combined with cold turkey 26 Tried in past pregnancy - not successful Walk/keep busy/slow wean or walks/reading 3 Willpower 2 Yoga, willpower, cold turkey 1 Don't know 9 Inadmissible 5 Total 297 The total number of responses did not correspond to the number of responses for attempts at cessation (n=411) as would be assumed. Inadmissible responses included answers that did not specify any method of cessation. 99 Summary Issues relevant to the integrity of the CTF section data include incomplete data, internal inconsistencies between related variables, as well as lack of specificity of various questions of the CTF. These factors constitute a major problem for the analyst who is unable to derive any definitive conclusions. Chapters Six and Seven provided results of the POP database from 1994/95. Each chapter addressed the two sections of the data; the IPRIT and CTF. The following chapter discusses the conclusions and recommendations of this study based on the results presented. 100 Chapter 8 CONCLUSIONS AND RECOMMENDATIONS This review of the 1994/95 Pregnancy Outreach Program (POP) database is intended to assess the reliability and validity of the data and the potential of the database for accessibility and use for reliable reporting, program evaluation and systematic epidemiological analysis. In this chapter, the maJor findings of this review are summarized and recommendations are made. The two sections of the database, the Individual Prenatal Risk Identification Tool (IPRIT) and Client Tracking Form (CTF) are discussed within the context of data quality issues. Individual Prenatal Risk Identification Tool (IPRIT) The intent of the IPRIT is to determine eligibility for the client to be enrolled in POP as well as to provide a basis for developing the care plan of the client. Risk assignment is one of the most important tasks the staff of POP perform. Coverage. Coverage is defined as the extent to which the database provides complete information about the population being studied. The issue of missing data constituted many problems in the assessment of this section of the database. Because of the lack of complete data for many of the risk factors, a complete and accurate description of the POP clientele in relation to physical, socio-economic, emotional and substance abuse factors cannot be established. Furthermore, any analysis of risks present in pregnancy, in relation to actual birth outcomes would be fraught with hazards because of the missing values. For example, the emotional risk factor section of the IPRIT had record completion rates which varied from 18.3% to 37.8% (refer to Table 3.13). At least 101 60% of the data was missing for all ofthe six variables measured in this section. Similarly, alcohol, cigarette, and drug use variables had low completion rates ranging from 22.4% to 61.5% (refer to Table 3.6). This renders the data set unacceptable since these variables are regarded as key factors in the risk screening process and are believed to be significant contributors to unfavourable outcomes. Further, since a reduction in substance abuse risk factors is a specific and high priority of the POP, the impact of client participation in the program cannot be assessed in this database. Physical risk factors had approximately the same completion rates as the substance abuse component of the IPRIT. These ranged from 22.3% to 61.8% (refer to Table 3.1) with the lowest completion rate being for PF8, multiple pregnancy. While it is true that the missing values may simply indicate that there were no medical or physical risk factors identified by the program staff, the variation in completion rates and the lack of definitive "none" or "zero" values gives little confidence that the medical and physical history of the client had been fully addressed. Socio-economic risk factors had the widest variation in completion rates that ranged from 24.5% to 85.8%. The lowest response rate was for SEF3, refusaVresistance to appropriate services, a response rate that undoubtedly reflects a "nil" value since it is unlikely that many clients would refuse services that were offered. Nevertheless, it is difficult to have confidence in a data set concerned with the maternal social environmental impact on birth outcomes when it has to be assumed that a missing value means that there has NOT been delayed access to prenatal care, that the housing is deemed to be adequate, and that the mother is not experiencing financial problems. One simply cannot be sure that these questions have been addressed at all by the staff 102 member. As indicated in this discussion of the IPRIT variables, it would be tempting to assume that missing values in a field constitute a true "no" or "zero." Nevertheless, imputing a "no" or "zero" to the incomplete fields for such relevant and salient variables would not be acceptable by any standards of database management. The risk of false negatives and their impact on descriptive or inferential analyses would be far too great to accept. Internal Consistency. As described in the methods section, recorded responses for the IPRIT were cross-tabulated to related variables in the CTF as measures of internal consistency. Many discrepancies were found in the recorded information provided. A notable example of this occurred in the companson of risk factor SEF1, single parenthood, to marital status information provided in the CTF (refer to Table 3.11 ). Of the 518 client files that had SEF 1 (i.e. single parenthood) indicated as a risk factor, only 355 matched the CTF marital status data. Cross-referencing PF12, age 17 and under or age 36 and over at time of delivery, with actual client ages provided in the CTF section (refer to Table 3.5) also showed wide discrepancies. Of the 248 records with PF12 indicated, only 132 matched the age data. Reasons for this are unknown and it is difficult to understand why such discrepancies should occur in the recording of basic variables such as age and marital status. The presence of these discrepancies does reflect on the quality of the database as a whole. Specificity of Risk Factor Definitions. An issue related to the reliability and validity of the recorded data arises from the definitions of the possible responses to a question contained in the IPRIT guide. Some definitions appear in two or more factors, leading to possible double counting in assessing risk. For example, some parts of the definition for 103 mental health problems (EF2) are duplicated in other risk factor criteria such as family history of abuse/neglect (EFl) and inability to cope regarding pregnancy and baby (EF4). Another issue concerning definitions of responses relates to the specificity with which the responses are defined. For example, the history of physical/medical conditions that define risk factor PF2, illness/condition with impact on pregnancy, provides an extensive list of factors but no clear criteria are stated for the inclusion or exclusion of the various conditions named. Client Tracking Forms {CTF) The CTF includes information about referral to program and client intake, demographic information about the POP clientele, client monitoring information related to substance abuse and other relevant factors, as well as pregnancy outcome information. The CTF, in essence, was intended to be the basis for the POP evaluation. Data quality issues relevant to this section of the database were similar to those identified in the IPRIT section. Missing values, incomplete records, and various inconsistencies in the recording of the data in this section precluded any analysis beyond the identification of problems in the inspection of the database. Problems encountered can be enumerated as follows. Coverage. Item-per-item completion rates for each question of the CTF were variable. Completion rates varied from 1.3% for a question asking whether referral for the client was made to mental health services to 100% for demographic questions such as age and ethnic background. Such wide variation makes it difficult to assess the capacity for the database information to form the basis of a valid program evaluation. It also raises questions as to whether this was due to lack of assessment on the recorder's part or 104 inadequate knowledge about the client. Section eight of the CTF had the lowest completion rates of this entire section of the database (refer to Table 11 ). It is speculated that this may be due to confusion on the recorder's part about how to complete this question if no referral was made for that client to a particular agency or community service. Ideally, ifthere was no referral made, then the answer should have been a figure of zero (i.e. "0") to indicate this and provide for complete data for this field. The various sections of the data evaluating nutrition had variable response rates. Food intake information which evaluated the number of servings from each food group (refer to Table 9.4) had very good response rates ranging from 82.1 % to 94.8%. In other sections, however, there were poorer response rates. In particular, Table 9.5, which presented the data related to intake of caffeine, sweetened drinks and water had wide variation in completion rates (5 .7% to 55 .8%). In addition, Table 9.6, which provided data related to intake of foods that were sources of folate and iron, had generally low response rates ranging from 22.7% to 70.0%. While it is uncertain why this section had such overall low rates of completion, it is evident that the data could not be analyzed with any confidence to evaluate the Program's contribution to client nutritional status in pregnancy and its relation to POP participation and birth outcome. Furthermore, the current information collected in this database to assess nutritional status needs to be critically examined as it provides little information about the client's personal goals in relation to nutrition and how these goals are achieved. Internal Consistency. There were discrepancies for certain questions in the CTF that should have equivalent numbers of recorded responses. For example, questions asking 105 about illicit drug use (refer to Table 9.9) at program intake revealed that seventy-eight types of drugs were being used on a weekly basis by a sub-population of the total clients who completed the program. Table 9.10, however, shows 301 responses for the types of drugs used. Part of the reason for this discrepancy is that some of the responses were not specific drugs per se and included responses such as alcohol and prescribed medications. Ascertaining why this variation occurs is difficult but because there is such a wide discrepancy, the quality of the data in relation to evaluating illicit drug use as it relates to the Program clientele is questionable. Another example of internal consistency which places the evaluation of substance abuse issues of this database into question relates to sections ten and eleven of the CTF (refer to Tables 12.5 to 14.2). In Table 12.5, it is indicated that alcohol use follow-up information is provided for 462 clients and that smoking follow-up data is provided for 544 clients. Responses to questions in sections ten and eleven of the CTF which provide the alcohol and smoking data respectively consistently do not add up to these figures. Specificity Of Definitions Provided For The CTF. Unlike the IPRIT, the CTF did not include a guide that outlined criteria for the various questions of the document. This could prove problematic for some questions such as the one asking for the client's body mass index (BMI). This measurement of pre-pregnancy body weight appropriateness can be calculated by a commonly accepted formula. Alternatively, it could also be assessed by various tables that provide values rounded off to one decimal place. Each method provides answers that can vary as much as 0.5, a difference that may prove relevant when attempting to determine if this is a presenting risk factor. Another key issue to address relates to how the information for the CTF IS 106 collected. The current system of collecting information for the CTF is not standardized in the form of specific questions and reference categories. The CTF is based on locally developed client charts. Therefore, the information cannot be collected in a consistent manner. Evaluating The Database For Evaluation And Research As described in Chapter One, the original intent of this thesis was to use an existing database on the Pregnancy Outreach Program of BC (POP) to develop and test a model of program evaluation and to examine the relationship between the identification of risk factors and actual birth outcomes. Based on an extensive literature review, a conceptual framework was developed to assess the database in terms of utility of program evaluation, decision making and policy development, and for epidemiological type research. Inherent in this approach was the expectation that the database would be accessible, analysable and complete, and the variables would have been coded and entered into the electronic database in a consistent and reliable fashion. At the time that development of the conceptual framework and exploration of the literature related to risk factors and birth outcomes was proceeding, work began on opening the database and preparing for the planned analyses. Technical differences were encountered in opening the database and in converting the software on which it was based to a software program that would more readily facilitate the type of analyses proposed. Once these difficulties had been overcome, and it was possible to run simple frequency tables, it became clear that there were inherent problems in the database itself including problems in defining the population of cases and establishing those cases that had a complete record (i.e. from program entry to birth outcomes and breastfeeding 107 practices). Further, and of even greater significance for the proposed analyses, all client records had a large number of variables for which a code or response in specified fields were missing. The percentage of missing responses has been fully documented in Chapters Six and Seven of this thesis and it is clear that, unless one takes the view that a "zero" or "nil" response can reliably be imputed to more than 50% of the cases, the results of any analyses, including descriptive analyses, cannot be accepted as providing a reliable and valid description of the population or of a relationship between any of the variables under study. Further, upon closer inspection and cross tabulation of variables, a lack of internal consistency in the recording of key variables was found . In addition, there was a lack of specificity both with respect to the definitions of the variables, the questions that elicited the responses, and the coding structures within the data collection forms . Explaining The Problems And Issues Inherent In The Data Base Chapter Four provided an extensive literature review of evaluating electronic databases intended for health program evaluation. Four key areas for reviewing such databases included examination of its utility for program evaluation, utility for health research, utility for decision-making and policy development as well as the selection of information system. Based on the literature review a conceptual framework was established and can be referred to in Figure 4 (refer to page 42). Attempts to apply this model to the assessment of the CTF section of the POP database were not attempted because of the issues of lack of completeness of data for the variables measured, internal inconsistencies within the database and lack of specificity of definitions of the questions asked within the database. For example, using the information available to determine the program's effectiveness in reducing adverse birth outcomes would involve numerous 108 unacceptable imputations of data and compromises that would result in a lack of confidence in the assessment provided. Further, when investigating the effectiveness of the database in providing adequate information coverage as outlined in the framework there were also shortcomings. For example, the database fails to provide sufficient data to evaluate the program's effectiveness in reaching subgroups of the POP clientele such as those who have been determined to be using alcohol or illicit drugs in pregnancy. These key variables are not measured or recorded in a valid or reliable form. This thesis has documented fully the problems encountered in the database with respect to every variable that is entered. The question then arises as to why, given that this database is intended to be the basis for program reporting and aggregation at the provincial level, for program evaluation, and for policy and decision making, the data is not systematically recorded and cannot be used with confidence for the purposes for which it was intended. The answer seems to lie in the fact that the POP database is an example of secondary data, a term defined by Glaser (1963) as "existing data which were originally gathered for other purposes." In the present case, the data were derived from records that were part of an ongoing service program and that formed the basis for the identification of risk factors for individual clients and for planning interventions and that documented the individual client's progress and outcomes. The variables, questions, responses, coding structures were all aimed at providing the service provider with information that was needed for the management of the individual client and for the implementation of the program for that client. In this sense, it appears that the step of adapting the information and the responses that were gathered 109 in the service context to creating an aggregated database that would meet the goals that were set for it were not taken when the electronic database format was adopted. In retrospect, it is perhaps not surprising that the database was developed from service provider records without the rigorous review of the variables and their coding structures that would be required for the purpose of program evaluation or epidemiological research. The need for such a review is not readily identified by program planners, policy makers, and service providers, especially if the set of questions that might be addressed from the database has not been fully specified and developed. Further, it is only in recent years, that health service providers have moved, and slowly at that, to electronic data based client or patient record keeping. Recommendations Faced with the decision to focus the thesis on the quality of the data in th: database, recommendations made in this chapter tend to be global in nature. The problems inherent in the database are so profound that using the database for reporting, policy making, and reporting is fraught with risks. Further, it seems questionable whether the costs and efforts involved in taking the records from the individual program sites and translating them into a provincial database is cost effective, whether at the program level or at the regional level. Finally, the task of developing a database that meets the objectives that were set for it would require a major commitment of time and resources to review the variables, the coding structures, and the methods by which the information is translated into an electronic base that can be used at all levels of the program and by researchers and planners. Brief comments are, however, made on some of the specific areas in order to focus the attention of those charged with the 110 responsibility for providing and evaluating this program. The Reliability And Validity Of Reports Based On Database. The results of this review suggest that reports based on this database must be viewed with appropriate caution. For example, the 1994/95 provincial report released in 1996 suggests that program outcomes were favourable, however, it does not provide companson of outcomes for those who did and did not complete the program. It is clear from this review of the data that such reports could provide, at best, only a broad picture as to the number of clients served and a limited number of characteristics, whether demographic, obstetric or medical. The broad picture reflected by the number of clients who are seen by the program is certainly a useful index of program activity and the fact that a record, imperfect as it may be, does exist gives some indication of the activity of the program. Nevertheless, the demographic data are incomplete and would not provide an accurate description of the population being reached. Similarly, all risk factors as recorded in the database have serious defects and it would not be appropriate to use the risk factors as record to reach any conclusions about the extent to which the needs of a high risk population are being met. One can certainly conclude that since the mothers have met the criteria for inclusion in the program, they are indeed at risk of unfavourable outcomes, but the degree and patterning of the risks cannot be easily documented. ~ And Client Identifiers. In its current form, the database provides an overall documentation of the program. If, however, it is to meet the goals that have been set for it, then there are a number of substantive issues that need to be addressed. First, it is questionable whether the present records that form the basis for the derivation of the variables (i.e. the IPRIT and the CTF) can, or even should be, changed or adapted. They 111 are currently the forms and questions with which the program workers are familiar and one would not question whether they fulfill the needs of those dealing with clients on a day to day basis. It is apparent, therefore, that considerable thought needs to be given to each and every variable that belongs in a program and provincial database in order to meet the overall goals, beyond those of the individual client and program worker. For example, in order to make the database consistent with other social and health databases, standard definitions of variables such as derived from Statistics Canada (e.g. the census in the case of demographic variables) should be adopted together with a common coding framework. This will allow comparison of the population of clients served with the general population and with other specific populations as well as permitting consistent comparison between programs in the province. While recognizing that there are problems of confidentiality, common record identifiers should be used in order that the database can be linked to other data files such as provincial vital statistics, and medical and hospital records. The Recording Of Risk Factors. It is beyond the scope of this thesis to comment on the appropriateness of the selection of the risk factors and medical and obstetric factors that should be entered into a database. It is apparent that IPRIT and other instruments are largely oriented towards "clinical" assessment rather than for rigorous program evaluation or epidemiological research. Indeed, the review of the literature suggests that there is a lack of agreement as to which risk factors impact definitively on birth outcomes and how these risk factors can be measured. For example, patterns of smoking, alcohol, and drug use are notoriously difficult to capture and, while some 112 improvement can be made in the current forms used in the POP, it is by no means clear that valid information can be elicited from this population. Similarly, the gathering of obstetric and medical data, even g1ven specific guidelines and categories, is difficult to do in a "clinical" practice context and it may be that only very specific data that elicits the presence or absence of salient medical and obstetric conditions should be incorporated into a province wide database. Notwithstanding these caveats, however, there is clearly a need for the systematic gathering of such data to better understand the relationship between the risks factors and birth outcomes. Whether the POP is the setting in which the answers may be provided is a question in itself. It may be that such queries are better addressed by "gold standard" clinical trials or in specific studies that may build on the POP clientele base. Recording And Entering The Data. In reviewing the electronic database, one must understand that "front line" workers engaged in providing a complex and demanding program are being asked to add another task to their job duties which, for the most part, they have not been prepared or trained. Even if the questions and coding structures were clearly specified, the task of transferring these to electronic format may be better left to those with special training and skills in this process. It is possible that the current forms that are being used could be adapted for direct computer assisted entry but this is hardly to be recommended given the need to completely review the variables contained within the forms. Ideally, in the next phase of database development of POP, consideration would be given to using a computer assisted data entry system, one that can be easily adopted by health care workers in the field. 113 Computer Software. The selection of appropriate software for database management and analysis is a controversial subject that cannot be addressed within the scope of this thesis. Indeed, any recommendations that were made would be outdated long before any new system was considered or put in place. As in the case of the variables in the database, the software should be such that it is compatible with other applications with which the database might link or at least be readily transformed to other software when linking is required or complex analyses have to be undertaken. The health care data systems industry is moving rapidly and it is hoped that the next iteration of the POP database will take advantage of the new developments. Summary And Conclusion This thesis began with the proposal to utilize an existing database of a provincial program, the Pregnancy Outreach Program (POP), to develop and implement a model of program evaluation and to examine the relationship between identified risk factors and outcomes. On opening the files, however, it was found that the number of incomplete data fields, inconsistencies within the database itself, and non specificity of responses precluded any attempt at analysis, even at the descriptive level. It was concluded, therefore, that it would be valuable to fully document the shortcomings in the quality of the data contained in the files since "feed back" on this central issue in program and evaluation could be central to modifying or even continuing the accumulation of data on the program. In this event, the result sections (Chapters Six and Seven) examined each variable that is contained in the database in detail. It is concluded that, for the most part, the 114 quality of the database does not permit it to be used for program description, evaluation, nor for outcome analysis. It is suggested that the reason for the problems inherent in the database lies in the fact that forms and instruments that are largely used for service staff working with individual clients are the source of the variables that are coded and entered. As a result, it is suggested that the database should be accepted as a record of service and clinical contacts but that its limitations should be clearly recognized in terms of providing a picture of POP at the local, let alone the regional or provincial level. It is suggested that consideration be given, if program evaluation and outcome research is to be undertaken on POP to develop a database separately that is oriented towards the evaluation and research functions. 115 REFERENCES 9. Alkin, M.C. (1969). Evaluation theory development. Evaluation Comment 2(1), 2- Amaro, H., Fried, L., Cabral, H. , & Zuckerman, B, (1990). Violence during pregnancy and substance abuse. American Journal ofPublic Health. 80(5), 575-579. Arbuckle, T., Sherman, G., Kawamoto, Y. , & Mers, A. (1989). Predictors of birth weight from the Nutrition Canada follow-up cohort. Pediatric and Perinatal Epidemiology. 3, 115-129. Backett, E.M. (1984). The risk approach in health care with special reference to maternal and child health. including family planning. Geneva, Switzerland: World Health Organization (Public Health Paper, No. 76). Barber, B., Davey, J. (1994). Approaching safe and secure health information systems in Europe. Computer Methods and Programs in Biomedicine. 44, 23-29. Measuring the impact of nursing Barriball, K.L. , Mackenzie, A. (1993). interventions in the community: a selective review of the literature. Journal of Advanced Nursing, 18, 401-407. Battista, R.N., Feeny, D.H., Hodge, M.J. (1995). Evaluation of the Canadian Coordinating Office For Health Technology Assessment. International Journal of Technology Assessment in Health Care, 11(1), 103-116. Behrman, R. (1985). Preventing low birth weight. Summary. Division of Health Promotion and Disease Prevention Institute of Medicine. Washington, DC: National Academy Press. Birkhead, G.S. (1992). Moving public health surveillance onto the health agenda: the perspective of a district health office. In: Wetterhall, S.F. (1992). Proceedings of the 1992 International Symposium on Public Health Surveillance. MMSR, 42(suppl), 29-35 . Bohn, D. (1990). Domestic violence and pregnancy. Journal of Nurse-Midwifery, 35(2), 86-98. Botting, G., MacFarlane, A., & Price, F. (1992). Three, four and more. A study of triplet and higher-order multiple births. London: HMSO. B.C. Ministry of Health, Prevention and Health Promotion, Nutrition Section (1995). Pregnancy Outreach Program 1994/95 status report. Victoria, B.C.: Ministry ofHealth. B.C. Ministry of Health and Ministry Responsible for Seniors, Community and Family Health Nutrition Branch (1993). Pregnancy Outreach Program Handbook. 116 B.C. Ministry of Health, Prevention and Health Promotion, Nutrition Section (1994). Pregnancy Outreach Program 1993/94 status report. Victoria, B.C. : Ministry of Health and Ministry Responsible for Seniors. B.C. Ministry of Health and Ministry Responsible for Seniors, Community and Family Health, Nutrition Branch (1993). Pregnancy Outreach Program. Qualitative Evaluation Report. Victoria, B.C. : Ministry of Health and Ministry Responsible for Seniors. B.C. Ministry of Health and Ministry Responsible for Seniors, Community and Family Health, Nutrition Branch (1994). Pregnancy Outreach Program. 1992/93 status report. Victoria, B.C.: Ministry of Health and Ministry Responsible for Seniors. B.C. Ministry of Health and Ministry Responsible for Seniors, Community and Family Health, Nutrition Branch (1993). Pregnancy Outreach Program. 1991/92 status report. Victoria, B.C.: Ministry ofHealth and Ministry Responsible for Seniors. B.C. Ministry of Health and Ministry Responsible for Seniors, Community and Family Health, Research and Evaluation (1990). Pregnancy Outreach Projects. Quantitative Evaluation Report. Victoria, B.C.: Ministry of Health and Ministry Responsible for Seniors. B.C. Ministry of Health and Ministry Responsible for Seniors, Prevention and Health Promotion (1995). Management of Client Data For Provincial Evaluation. Victoria, B.C.: Ministry ofHealth and Ministry Responsible for Seniors. B.C. Ministry of Health (1988). Guidelines for Coordinators of Pregnancy Outreach Programs (Draft). Victoria, B.C.: Ministry ofHealth. Bobadilla, J.L. (1992). Evaluation of maternal health programs: approaches, methods and indicators. International Journal of Gynecology and Obstetrics. 38(suppl), S67-S73. Boyle, M.H., Torrance, G.W. , Sinclair, J.C. , Horwood, S.P. (1983). Economic evaluation of neonatal intensive care of very-low-birth-weight infants. New England Journal ofMedicine. 308, 1330-1337. Campbell, J., Poland, M., Waller, J., Ager, J. (1992). Correlates ofbattering during pregnancy. Research in Nursing and Health. 15, 219-226. Caro, F.G. (1977). Readings in evaluation research (2nd edition). Russell Sage Foundation. New York: Chomitz, V., Lieberman, E., Cheung, L. (1992). Healthy mothers - Healthy beginnings. A white paper. Boston: Harvard School of Public Health. 117 Clark, J. (1983). Evaluating health visiting practice. Health Visitor. 55(6), 205-208 . Code, D. (1995). Review of the Nutrition Component of the Pregnancy Outreach Program. A Survey of POP Coordinators. Victoria, B.C.: Ministry of Health, Prevention and Health Promotion Branch, Nutrition Section. Cohen, M ., Mac William, L. (1995). Medical Care. 33(12), DS21-DS42 . Measuring the Health of the Population. Corman, H., Grossman, M. (1985). Determinants of neonatal mortality rates in the U.S. Journal ofHealth Economics. 4, 213-236. Cowley, J. (1986). When health promotion works, the opposition begins: personal view. Health Promotion. 1, 201-209. A Creasy, R.K. (1993). Preterm birth prevention: Where are we? American Journal of Obstetrics and Gynecology. 168, 1223-1230. Creasy, R.K., Merkatz, I.R. (1990). Prevention of Preterm Birth: Clinical Opinion. Obstetrics and Gynecology. 76(S1), 2S-4S. Creasy, R.K., Gurnmer, B.A. , Liggins, G.C. (1980). System for prediction spontaneous preterm birth. Obstetrics and Gynecology, 55, 692-695 . Cronbach, L.J. (1982). Designing evaluation of educational social programs. San Francisco: Jossey-Bass. Curran, E. (1993). Help From Epi Info. Nursing Times, 89(29), 66-67. De Vries, H., Weijts, W., Dijkstra, M. , Kok, G. (1992). The Utilization of Qualitative and Quantative Data for Health Education Program Planning, Implementation, and Evaluation: A Spiral Approach. Health Education Quarterly, 19(1), 101-115. Dean, A.D. , Dean, J.A. , Burton, A.H., & Dicker, R.C . (1990). Epi Info Version 5: a word processing database and statistics program for epidemiology on micro-computers. USD, Incorporated, Stone Mountain, Georgia. Denver, T. (1994). An overview of software assessment. Computer Methods and Programs in Biomedicine, 44, 5-60. Desrosiers-Choquette, J., Julien, M. (1995). Prenatal Nutrition and Support Programs: Making Evaluation Work For You - Indicators For the Canada Prenatal Nutrition Program. Ottawa, Ontario: Health and Welfare Canada. 118 Dissevelt, A.G. (1976). An antenatal record for identification of high risk cases by auxiliary midwives at rural health centres. Tropical and geographical medicine. 28, 251255. Dobbyn, B. (1993). Communication Strategy and Computer Specifications Pregnancy Outreach Program. Victoria, B.C. : Ministry of Health and Ministry Responsible for Seniors, Family Health. Dobbyn, B. (1992). Pregnancy Outreach Program Procedures. Victoria, B.C.: Ministry of Health and Ministry Responsible for Seniors, Family Health. Donaldson, D., Winquist, P. (1995). Within Our Reach Perinatal Outreach Training Manual. Burnaby, B.C.: Douglas College. Drennan, V. (1990a). Gathering information from the field. Nursing Times 86(39), 46-48. Drennan, V. (1990b). Striving for fairer workloads. Nursing Times. 86(40), 48-49. Duncan, K. (1979). Information Technology and Health Care - The Critical Issues. Virgina: AFIPS Press. Eisner, E. (1979). The educational imagination. New York: Macmillan, 1979. Elster, A.B., Lamb, M.E., Tavare, J. , Ralston, C.W. (1987). The Medical and Psychosocial Impact of Comprehensive Care on Adolescent Pregnancy and Parenthood. Journal ofthe American Medical Association. 258(9), 1187-1192. Essex, B.J., Everett, V.J. (1977). Use of an action oriented record card for antenatal screening. Tropical Doctor. 7, 134-138. Fairburn, R., Dhanani, S. (1993). Pregnancy Outreach Program's Client Database Project User's Manual. Victoria, B.C.: Ministry ofHealth. Feinleib, M. (1993). From Information to Knowledge: Assimilating Public Health Data. American Journal of Public Health. 83(9), 1205-1207. Ferri, F., Grifoni, P ., Meo-Evoli, L. , Pisanelli, D., Ricci, F. (1993). ADAMS: Aggregate Data Management System for epidemiologists and health-care managers. Computer Methods and Programs in Biomedicine. 40, 43-53. Frohlich, N. , Markesteyn, T., Roos, N., Carriere, K., Black, C. , DeCoster, C., Burchill, C., & MacWilliam, L. (1995). Stability and Trends Over 3 Years of Data. Medical Care. 33(12), DS100-DS108. 119 Geronimus, A.T. (1986). The effects of race, residence, and prenatal care on the relationship of maternal age to neonatal mortality. American Journal of Public Health. 69(7), 653-657. Gibsons, R. (1989). University of Guelph. Principles of Nutritional Assessment. Guelph, Ontario: Ginzberg, E. (1991). Health Services Research - Key to Health Policy. Massachusetts: Harvard University Press. Glaser, B.G. (1963). Retreading research material: the use of secondary analysis by the independent researcher. American Behavioural Scientist. 6, 11-14. Goeppinger, J. (1988). Challenges in assessing the impact of nursing service: a community perspective. Public Health Nursing. 5(4), 241-245 . Gold, R.B., Kenney, A.M. , Singh, S. (1987). Paying for maternity care in the United States. Family Planning Perspectives. 19, 190-206. Goldenberg, R.L., Davis, R.O ., Copper, R.L. (1990). The Alabama preterm birth prevention project. Obstetrics and Gynecology. 75 , 933-939. Goodwin, J.W. , Dunne, J.T. , Thomas, B.W. (1969). Antepartum identification ofthe Fetus At Risk. Canadian Medical Association Journal, 101 , 58. Graham, W.J. , Filippi, V.G.A. (1994). Monitoring Maternal Health Goal: How Well Do the Indicators Perform? Maternal and Child Epidemiology Unit Publication No. 2. London: London School of Hygiene and Tropical Medicine. Guba, E.G., Lincoln, Y.S. (1981). Effective evaluation. San Francisco: JesseyBass. Hakim, C. (1982). Secondary Analysis in Social Research. London: George Allen and Unsin. Hamilton, G.A. (1993). An Overview of Evaluation Research Methods with Implications for Nursing Staff Development. Journal of Nursing Staff Development. .2.(.3.), 148-154. Hart, R.H. (1977). Maternal and child health services in Tanzania. Tropical Doctor, l , 179-185 . Hauchecorne, C.M. (1995). "POPCORN" (Pregnancy Outreach Program ClientOriented Record About Nutrition) - An Approach to Evaluate Nutrition Services Delivered Through the Pregnancy Outreach Program in British Columbia. Victoria, B.C.: Ministry of Health and Ministry Responsible for Seniors. 120 Health Canada (1997). Canada Prenatal Nutrition Program National Evaluation Strategy Information Package. Calgary, Alberta: Gail V. Barrington and Associates, Inc. Health and Welfare Canada (1981). Report of the Federal Task Force on High Risk Pregnancies and Prenatal Record Systems. Ottawa, Ontario: Health and Welfare Canada. Heins, J.C., Nance, N.W., McCarthy, B.J., Efird, C.M. (1990). A randomized trial of nurse-midwifery prenatal care to reduce low birth weight. Obstetrics and Gynecology. 75 , 341-345 . Herron, M.A. , Katz, M., Creasy, R.K. (1982). Evaluation of a preterm birth prevention program: preliminary report. Obstetrics and Gynecology. 59, 452-456. Hollyer, J. (1991). Epi Info Version 5: a review. PHLS Microbiology Digest, 8(2), 60-62. Hueston, W.J., Knox, M.A. , Eilers, G., Pauwels, J., Lonsdorf, D. (1995). The Effectiveness ofPreterm-Birth Prevention Educational Programs For High-Risk Women: A Meta-Analysis. Obstetrics and Gynecology, 86(4), 705-711. Hueston, W.J. (1992). A critical assessment of preterm labor prevention strategies. The Journal of the Family Practice, 35(1), 81-89. Institute of Medicine, National Academy of Sciences (1978). Assessment of Health Services Research, A Working Paper. Washington, D.C. : National Academy Press. Institute of Medicine (1988). Prenatal Care: Reaching Mothers, Reaching Infants. Washington, DC: National Academy Press. Institute of Medicine (1985). National Academy Press. Preventing low birth weight. Washington, D.C.: Kay, B.J., Share, D.A. , Jones, K. , Smith, M., Garcia, D., Yeo, S.A. (1991). Process, Costs and Outcomes of Community-Based Prenatal Care for Adolescents. Medical Care, 29(6), 531-542. Institute of Medicine, National Academy of Science, Committee On Scientific Evaluation of WIC Nutrition Risk Criteria, Food and Nutrition Board (1996). Summary of WIC nutrition risk criteria: A scientific assessment. Journal of the American Dietetic Association, 96(9), 925-30. Konte, J.M., Creasy, R.K. , Laros, R.K. (1988). California north coast preterm birth prevention project. Obstetrics and Gynecology, 71 , 727-730. 121 Lennox, C.E. (1981). An action oriented antenatal record for Papua New Guinea. Papua New Guinea Medical Journal, 24, 280-285. Lesinki, J. (1975). High Risk Pregnancy. Obstetrics and Gynecology, 46(5), 599. Lipshitz, J., Pierce, P., & Artnz, M. (1993). Preterm labour. Philadelphia: W.B . Saunders Company. Maine, D.M ., Richardson, D.K., Hadley, C., Gabbe, S.G. (1989). Can preterm deliveries by prevented? American Journal of Obstetrics and Gynecology, 151 , 892-898 . Maine, D.M., Richardson, D.K. , Hadley, C.B., Gabbe, S.G. (1989). Controlled trial of a preterm labor detection program: efficacy and costs. Obstetrics and Gynecology, 74, 873-877. Martin, C., Armstrong, R. (1995). The Impact of Prenatal Behaviour Modification On Maternal and Infant Outcomes: British Columbia's Pregnancy Outreach Program. Vancouver, B.C.: University ofBritish Columbia. Moore, T.R. , Origer, W., Key, T.C. (1986). The perinatal and economic impact of prenatal care in a low-socioeconomic population. American Journal of Obstetrics and Gynecology, 154(1), 29-33 . Mora, J.L. (1979). Nutritional Supplementation and the Outcome of Pregnancy. Nutrition Reports Interviews, 18, 167. Morrison, J.C. (1990). Preterm Birth: A Puzzle Worth Solving. Obstetrics and Gynecology, 76(S 1), 5S-12S . Mortimer, G. , Merkatz, I.R. , Hill, J.G. (1991). Caring for our future : A report by the Expert Panel on the content of prenatal care. Obstetrics and Gynecology, 77(5), 782-787. Naismith, D.J. (1980). Maternal Nutrition and the Outcome of Pregnancy - A Critical Appraisal. Proceedings of the Nutrition Society, 39, 1. Norris, F.D., Williams, R.I. (1984). Perinatal outcomes among Medicaid recipients in California. American Journal ofPublic Health, 74(10), 1112-1117. Nutbeam, D., Smith, C., Murphy, S., Catford, J. (1993). Maintaining evaluation designs in long term community based health promotion programmes: Heartbeat Wales case study. Journal of Epidemiology and Community Health, 47, 127-133 . Papiernik, F. , Bouyer, J. , Yaffe, K. , Winisdorffer, G. , Collin, D. , Drefus, J. (1986). Woman's acceptance of a preterm birth prevention program. American Journal of Obstetrics and Gynecology, 168, 939-946. 122 Parents of Multiple Births Association, (1991). Impact of multiples on the family: A random survey. Toronto: Parents of Multiple Births Association (POMBA). Patton, M.Q. (1980). Qualitative evaluation methods. Beverly Hills, California: Sage Publications, Inc. Peoples, M.D., Grimson, R.C., Daughtry, G.L. (1984). Evaluation of the Effects of the North Carolina Improved Pregnancy Outcome Project: Implications for State Level Decision Making. American Journal of Public Health. 74(6), 549-554. Pearson, A. (1987). Nursing Quality Measurement. Chichester: John Wiley and Sons. Polit, D.F., Hungler, B.P. (1991). Philadelphia: J.B. Lippincott Co. Nursing research: Principles and methods. Province of British Columbia. Closer To Home - The Report Of The British Columbia Royal Commission On Health Care and Cost. Volume 2. Victoria, B.C. : Ministry of Health and Ministry Responsible for Seniors. Raeburn, J.M., Rootman, I. (1989). Towards an expanded health field concept: conceptual and research issues in a new era of health promotion. Health Promotion. 3, 383-392. Rantanen, J. (1995). Evaluation of research - a difficult but necessary task. Scandinavian Journal ofWork. Environment and Health. 21 , 321-324. Reed, R. (1991). Secondary data in nursing research. Journal of Advanced Nursing, 17, 877-883. Rippey, R.M. (1973). McCutchan. Studies in transactional evaluation. Berkeley, California: Roos, N., Shapiro, E. (1995). Using the Information System to Assess Change: The Impact ofDownsizing the Acute Sector. Medical Care. 33(12), DS109-DS126. Rossi, P.H. , Freeman, H.E. (1989). Evaluation: A systematic approach (4 1h edition). Newbury Park, California: Sage, 1989. Rubin, A., Babbie, E. (1993). Research Methods for Social Work, Second Edition. USA: Brooks/Cole Publishing Company. Rush, D. (1981). Nutritional Services During Pregnancy and Birthweight: A Retrospective Matched Pairs Analysis. Canadian Medical Association JournaL 125, 567. 123 Savitz, D.A., Blackmore, C.A., Thorp, J.M. (1991). Epidemiologic characteristics of preterm delivery; Etiologic heterogeneity. American Journal of Obstetrics and Gynecology. 164, 467-471. Schlesinger, M., Kronebusch, D. (1990). The failure of prenatal care policy for the poor. Health Affairs. 9(4), 91-111. Selzer, M.L. (1971). The Michigan Alcohol Screening Test: the quest for a new diagnostic instrument. American Journal ofPsychiatry. 127, 1653-1658. Shah, K.P ., Shah, P.M. (1981). The mother's card: a simplified aid for primary health workers. WHO Chronicale. 35, 51-53. Shah, K.P. (1978). Surveillance card for married women for better obstetric performance. Journal of Obstetrics and Gynecology oflndia. 28, 1015-1020. Sheaff, R. , Peel, V. (1995). Managing Health Service Information Systems - an introduction. Pennsylvania: Open University Press. Smith, M.L., Glass, G.V. (1987). Research and evaluation in education and the social sciences. Englewood Cliffs, NJ: Prentice-Hall, Inc. Sokol, R.J. (1988). Finding the Risk Drinker in Your Clinical Practice. In Alcohol and Child/Family Health: Proceedings of a Conference with Particular Reference to the Prevention of Alcohol-Related Birth Defects, edited by Robinson, G., and Armstrong, R. , Vancouver, B.C. Spencer, G., Thomas, H., Morris, J. (1989). A randomized controlled trial of the provision of a social support service during pregnancy: The South Manchester family worker project. British Journal of Obstetrics and Gynecology. 96, 281-288 . Stecher, B.M. , Davis, W.A. (1987). How to Focus on Evaluation. Newbury Park, California: Sage Publications, Inc. Stuffelbeam, D.L., Foley, W.H., Gephard, W.L. , Guba, E.G., Hammond, R.I., Merriman, H.O., Provus, J. (1971). Education evaluation and decision-making. Itasca, Illinois: Peacock. Susser, M. (1981). Prenatal Nutrition, Birthweight and Psychological Development: An Overview of Experiments, Quasiexperiments and Natural Experiments in the Past Decade. American Journal of Clinical Nutrition. 34, 784. Susser, M. (1993). Health as a human right: an epidemiologist's perspective on the public health. American Journal ofPublic Health. 83 , 418-426. Task Force Report, Canadian Task Force on the Periodic Health Examination. 124 Canadian Medical Association Journal. 121 , November 3, 1979. The Institute of Child Health (1993). Best Start Communtiy Action for Healthy Babies. Prevention of Low Birthweight in Canada: Literature Review and Strategies. Ottawa, Ontario: The Institute of Child Health. The Institute of Child Health (1992). Prevention of Low Birth Weight in Canada: Literature Review and Strategies. Ottawa, Ontario: The Institute of Child Health. Thacker, S.B., Stroup, D.F. (1994). Future Directions for Comprehensive Public Health Surveillance and Health Information Systems in the United States. American Journal ofEpidemiology. 140(5), 383-397. Thompson, J. (1990). Maternal stress, anxiety and social support during pregnancy. Possible directions for prenatal intervention. In: Merkatz, 1., Thompson, J. (1990). New perspectives on prenatal care. New York: Elsevier. Thompson, J.C. (1992). Program Evaluation With a Health Promotion Framework. Canadian Journal ofPublic Health, 83 , S67-S71. U.S. Department of Health and Human Services, Public Health Service (1992). Improving Minority Health Statistics. Report of the Public Health Service Task Force on Minority Health Data. Washington, DC: U.S. Department of Health and Human Service, Public Health Service. Designing and Varkevisser, C.M., Pathmanathan, 1. , Brownlee, A. (1991). Ottawa, Ontario: International Conducting Health Systems Research Projects. Development Research Centre. Warnyka, J. (1979). Canadian Nurse, 75 , 18. Healthiest Babies Possible. The Vancouver Project. The Warrick, L.H., Wood, A.H. , Meister, J.S., De Zapien, J.G. (1992). Evaluation of a Peer Health Worker Prenatal Outreach and Education Program For Hispanic Farmworker Families. Journal of Community Health, 7(1), 13-26. Weiderhold, G. (1981). Databases for Health Care. New York: Springer-Verlag. Wilson, R.W., Schifrin, B.C. (1989). Is any Pregnancy Low Risk? Gynecology, 55(5), 653 . Obstetrical World Health Organization (1986). The growth chart. A tool for use in infant and child health care. Geneva, Switzerland: World Health Organization. 125 World Health Organization (1994). Home-based maternal records - Guidelines for development. adaptation and evaluation. Geneva, Switzerland: World Health Organization. Worthington-Roberts, B., Klerman, L. (1990). Maternal nutrition. In: Merkatz, I. , Thompson, J. (1990). New perspectives on prenatal care. New York: Elsevier. Wong, H. (1984). Micro and macro statistical/scientific database management. In: Proceedings IEEE International Conference on Data Engineering, 31-34. Los Alamitos: IEEE Computer Society Press. YWCA of Vancouver Crabtree Comer (1993). The Early Childhood Research Institute On Substance Abuse Challenges The Myth of Prenatal Substance Abuse. Vancouver, BC: YWCA ofVancouver. 126 APPENDIX A Province of British Columbia Individual Prenatal Risk Identification Ministry of Health Client ID# _ _ _ _ _ __ Date: Location _______ Explanation Yes Description Code Physical Factors Previous Pregnancy Loss Illness/Condition with Impact on Pregnancy Pre-pregnancy Weight (BMI) PF3 Rate of Weight Gain PF4 PF5 Inadequate Nutrition Previous Child With Anomaly PF6 Previous Child Requiring Neonatal PF7 Intensive Care Multiple Pregnancy PF8 Birth Interval PF9 PFlO Grand Multipara - 5 or more pregnancies PFll Established Genetic Risk PF12 Age Under 18/0ver 35 Socio-Economic Factors PFl PF2 SEFl SEF2 SEF3 Single Parenthood Delayed Access to Prenatal Care Refusal of/Resistance to Appropriate Services SEF4 Isolation - Ethnic, Language and/or Social SEF5 Limited Learning Ability/Illiterate SEF6 Unstable Relationship SEF7 Inadequate Housing SEF8 Financial Problems Substance Abuse Cigarette Smoking Alcohol Use Inappropriate Use of Over The Counter or Prescription Drugs SA4 Illegal Drugs Emotional Factors SAl SA2 SA3 EFl EF2 EF3 EF4 EF5 EF6 Family History of Abuse/Neglect Mental Health Problems Low Self-Esteem Inability to Cope/Anxiety Regarding Pregnancy and Baby Unrealistic Expectations Unwanted Pregnancy 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 White copy- headquarters Yellow copy- coordinator A guide for the use of Individual Prenatal Risk Identification Purpose The purpose of this guide is to simplify the process of identifying pregnant women for preventive service intervention. It is intended to complement the assessment of the physician by highlighting lifestyle factors in particular. The guide is not meant to be an all inclusive source of information of risks in families and pregnancies. It compiles in a single document basic information to assist professionals in the early identification of risks with the ultimate goal of reducing perinatal morbidity and mortality. Personal experience, knowledge and intuition on the part of the professionals are as important, if not more, than whatever guide or form is used. The guide should be used with the knowledge and understanding of risks, situations and their effect on health to arrive at a decision for appropriate intervention. The comprehensive multidisciplinary approach to care should be a sound principle to adopt. It will ensure that all points of intervention are covered and appropriate preventive measures are taken through community outreach and other family health programs of the health agency. The lists of risk factors noted on the forms are not meant to be all inclusive. They are intended to cover the most frequent problems producing risk. DEFINITIONS In general, the risk factors that will increase the chances of morbidity and mortality are of a physical, nutritional, mental/emotional, socio-economic or occupational nature. For the purpose of this guide, the following definitions have been adopted: risk: high risk groups: high risk families: high risk pregnancy: high risk infant: an increased probability of adverse outcomes groups with increased probability of adverse outcomes families whose circumstances indicate high risk factors which may interfere with optimum family life and functioning a pregnancy in which the mother and/or the fetus has an increased probability of maternal and fetal morbidity or mortality prenatally and intranatally newborn or infant with familial, maternal and perinatal factors that may lead to an increased morbidity and subsequent disabilities Physical Factors Previous pregnancy loss D Illness/condition with impact on pregnancy D Pre-pregnancy weight-body mass index (BMI) D Rate of weight gain D Inadequate nutrition D Previous child with anomaly D Previous child requiring neonatal intensive care D Multiple pregnancy D Birth interval D Grand multipara - 5 or more pregnancies 0 0 Established genetic risk Age under 18/over 35 D Previous pregnancy loss Previous pregnancy loss; habitual abortion, stillbirth, neonatal and infant death, such as SIDS are significant indicators. Depending on the cause of such loss the same conditions may be either present or occur again for another reproductive loss. The level of risk depends on the causative factor. Illness/condition with impact on pregnancy Many conditions may lead to premature labour, congenital anomalies, intrauterine growth retardation, and other associated morbidities. These include infections (rubella, STD, toxoplasmosis, genital herpes), abnormal presentation, surgical procedures during pregnancy, uterine and associated malformations, toxemia, anemia, bleeding, diabetes, hypertension, obesity, renal disease, isoimmunization etc. The risk and its effects are related to the severity of the condition. Other conditions such as blindness, deafness and physical handicaps can affect the mother in pregnancy. Depending on the individual's abilities, compensating mechanisms, support structure these conditions may be described as major, minor or no problems. Pre-pregnancy weight • • major- B.M.I. 18 and under or 29 and over BMI = wt (kg) ht(m 2) minor- BMI 18.1 to 20 or 27-28.9 Pre-pregnancy weight: The underweight woman has a BMI 18 or under. A BMI of 29 or over indicates obesity. A woman's nutritional status prior to and during pregnancy are important factors that influence the health of the fetus and the baby. The mother's pre-pregnancy weight and weight gain during pregnancy are two factors which affect the infant's birth weight and thus the infant's health. Rate of weight gain • if weight gain is 1 kg or less per month in 2nd and 3rd trimester if weight gain is less than 4 kg before 20 weeks if weight gain is greater than or equal to 3 kg/month after 12 weeks (sustained gain) Rate of weight gain: (For women beginning pregnancy at an acceptable BMI ie. 20 .1 to 26.9). Either excessive weight gain (3 kg or more per month after 12 weeks) or inadequate weight gain (1 kg or less per month in second or third trimester or less than 4 kg before 20 weeks) are all significant factors. Inappropriate rate of weight gain may lead to low birth weight infants and related problems. Inadequate Nutrition • • major - Consistently missing one or more food groups. minor- Consistently less than 50% of the recommended servings of one or more food groups. The 50% levels are : 3 servings of Breads and Cereals 3 servings of Fruits and Vegetables 1.5 servings of Milk and Milk Products 1 serving of Meat, Fish, Poultry and Alternates Inadequate Nutrition: The B.C. Food Guide for Pregnancy (see Baby's Best Chance pp. 22-26) outlines the appropriate numbers of food groups servings for adequate calories and nutrients. A deficiency can present a serious risk to the development of the fetus and to the mother's health. Previous child with anomaly • major- significant impact on development of the child eg. cyanotic heart disease, neural tube defects, cleft palate • minor- little impact on child's growth or development (readily corrected or only minor functional impairment) eg. minor orthopedic abnormalities, uncomplicated pyloric stenosis, vent-septal defects with spontaneous closure Previous infant with an anomaly or disorder: cerebral palsy, mental retardation, congenital anomalies ... if the same perinatal conditions still exist, they may lead to the same risk in the present pregnancy. Previous child requiring neonatal intensive care • • major- premature/postmature infant; low birthweight (1500 grams) minor- resolveable problems with no long range implications. N.B. Do not include previous child with may have been in care for 24 to 48 hours for supervision only. Multiple Pregnancy • major- pregnancies with more than one fetus Multiple Pregnancy: Prenatal mortality resulting from twin births is as high as 14%, the greatest mortality resulting from premature birth. There is conflicting evidence as to whether multiple gestation increases the nutrient and energy requirements above singleton pregnancies or whether the body utilizes nutrients more efficiently. Regardless of these conflicts to promote a normal pregnancy outcome, each diagnosis of multiple gestation is important and will need special nutrition intervention. Birth Interval • minor - less than 2 years. Although the optimum birth interval has not been defined, the incidence of fetal growth retardation and prematurity is consistently high when the birth interval is less than two years. Spacing allows time for the mother's body to recover and to be in optimal health before becoming pregnancy again. Grand multipara • fifth pregnancy or over Grand multipara: Parity alone combined with maternal age is significant. Higher risk of morrbidity occurs at the first pregnancy and at the fifth pregnancy or over. Established genetic risk • minor - previous pregnancy or present history Established genetic risk either from previous pregnancies or from a familial history ie. muscular dystrophy, cystic fibrosis, etc. is significant. Age under 18 over 35 Age- Under 18 years risk low birth weight. Over 35 risk chromosomal abnormalities. Socio-Economic Factors Single parenthood Delayed access to prenatal care Isolation - ethnic, language and social Limited learning ability/illiterate Unstable relationship Inadequate housing Financial problems 0 0 0 0 0 0 0 Social Environment: According to Knuppel and Drukker, the effects of maternal social environment on the outcome of pregnancy are recognized to be both multiple and profound. "Social environment" itself is described as the summation of numerous factors, including the family's standards of health and hygiene, housing and financial status, emotional and social support and so on. The effects may be direct or indirect and may be difficult to separate within the context of socio-economic status. It is the inter-relationship of these factors, rather than any single factor, that works to affect the outcome of the pregnancy. Single Parenthood • • major- when associated with multiple social problems minor - when financial and emotional support present Single parenthood. The frequency of cases of low birth weight infants and the perinatal mortality rates of infants born to unmarried mothers is double those of the children of married women, according to Knuppel and Drukker. Marital status alone is not necessarily an indicator of potential risk for mother and fetus so much as it is an indicator of an unwanted/unplanned pregnancy. These pregnant women, especially if unwed or teenagers, tend to neglect antenatal care and leave advice unheeded. Statistically, pregnancy complications occur more frequently in unmarried than in married women. Delayed access to prenatal care • • major- no care by 30 weeks minor - no care by 20 weeks Early access to medical care and return follow-up visits are essential for risk identification and monitoring. Some of the factors to consider are no medical care by 20 weeks, frequent missed appointments, no follow-up on medical advice and no attendance at prenatal classes in a pnm1para. Refusal of/resistance to appropriate services • • • major- refusal of appropriate professional services eg. MSSH social worker, etc. minor - resistance to appropriate services, accepts services with reluctance, may not heed advice Refusal of/resistance to appropriate services poses obvious threats to the client's receiving appropriate medical care and support for the mother and the fetus. Isolation - Ethnic, language and/or social • • major - total or severe isolation minor- isolated but with some support services in place Ethnic or language isolation can tend to deprive mothers of available information and resources. Social isolation ie. lack of supports, possibly new to area, can create a void in resources either classes or physicians which can put a mother at risk of not being assessed early and receiving adequate care and attention. Social isolation can create stress in the pregnancy. Limited learning ability/illiterate • • major- severe communicative disability minor - limited ability to understand Limited learning ability/illiterate especially if associated with other risks is significant. These people may not have access to information nor an understanding of the importance of education re. pregnancy and childbirth and child care. Marital problems, unstable relationship • minor - lack of support or discord Marital problems, unstable partnership: Marital discord, lack of partner support, lack of extended family support may lead to a higher incidence of reproductive loss, low birth weight (preterm, small for dates) nutritional problems, absence of maternal child bonding, neglect and abuse resulting in developmental delays and other associated morbidities. Inadequate housing • minor - lack of facilities, space, hazardous living condition, etc. Financial problems • minor - low income, unemployed, on assistance Unemployment, very low income, receiving social assistance may lead to a higher incidence of reproductive loss, low birth weight, nutritional problems, neglect and abusing resulting in developmental delays and other associated morbidities. Substance Abuse Cigarette smoking Alcohol use Inappropriate use of over the counter and Rx drugs Illegal drugs 0 0 0 0 Smoking • • major -more than 10 cigarettes/day minor- less than 10 cigarettes/day, daily secondhand smoke Cigarette smoking has been shown to decrease infant birthweight and increase the risks of perinatal morbidity and mortality. The growth-retarding effect of cigarette smoking and higher incidence of spontaneous abortions, stillbirths and placental complications among women who smoke during pregnancy may be due to several factors including direct toxicity of carbon monoxide, nicotine and/or other constituents of tobacco, reducing blood flow to the uterus affecting transfer of nutrients to the fetus , or suboptimal maternal food intake. Passive smoking may also be a cause of concern during pregnancy due to the oxygen-depleting effect of carbon monoxide. Alcohol use • • major- 4 or more drinks/day or binge drinking minor- 1-3 drinks/day Alcohol use: there is no known safe level of alcohol consumption for pregnancy women. It is not possible at this time to say what is the minimum level of alcohol consumption that may endanger the fetus. Chronic alcohol abuse (maternal alcoholism and malnutrition) may lead to the fetal alcohol syndrome; mental retardation, facial congenital anomalies, developmental delays, hyperactivity, etc. Cigarette smoking and heaving drinking (3 or more drinks per day) can independently increase the risk of spontaneous abortion and low birth weight infants. When both habits are combined, fetal risk is greatly decreased. Inappropriate use of over the counter and Rx drugs • • major- constitutes a hazard to pregnancy. Drug has mutagenic or tetratogenic effect minor - other drugs/herb use Illegal drugs • • major- any parenteral drug use; daily use of "soft" street drugs minor- regular use of "soft" drugs eg. marijuana Drugs may affect the intake, absorption, metabolism and/or utilization of nutrients in the body, thereby influencing maternal nutrition status. The effect that a drug has on the fetus depends on many factors including the type of drug, the amount taken by the mother, the stage of pregnancy at which it is taken and the frequency and duration of its use. Some drugs are known to have or strongly suspected of having an embryotoxic effect in humans (thalidomide, androgenic hormones, alcohol, anticonvulsants, isotretinoin, oral hypoglycemics). Others may possibly be embryotoxic in humans (female sex hormones, lithium, tranquilizers, anti-malarials, salicylates). Emotional Factors Family history of abuse/neglect Mental health problems Low self-esteem Inability to cope/anxiety regarding pregnancy and baby Unrealistic expectations Unwanted pregnancy/denial of pregnancy 0 0 0 0 0 0 Family history of abuse/neglect • classify as major if risk is severe: ie. recent enough to still affect emotional or physical health: or if possibility of repetition during pregnancy or shortly thereafter Family history of abuse/neglect (emotional or physical) tends to repeat itself from generation to generation and where there is abuse present in the home, the new baby is in high risk of being abused and neglected Mental Health Problems • • major - present psychiatric or mental health problems minor - history of psychiatric or mental health problems Mental health problems may shed light on ones family background, coping mechanisms, selfesteem and reactions to loss or crisis. As the pregnant woman strives to develop a degree of comfort with the many changes in social context of psychologic equilibrium, there often occurs a surfacing of conflicts that were never adequately resolved in earlier developmental periods. For example, pregnancy patients may experience conflicts of autonomy with their mothers, renewed rivalry with siblings, or active uncertainty about sexuality and disturbing fantasies about past relationships, each of which has been adequately deal with prior to pregnancy but which now result in troubling family interactions or marital discord. Manifest problems in adjustment prior to pregnancy, such as marital discord, economic difficulties, poor self-concept, and neuroticism may be exacerbated by pregnancy. Anxiety allowed to go unallayed may lead to maladaptive mother-child interaction. Low self-esteem • minor - lack of self-worth and motivation, depressed, uncaring, etc. Low self-esteem can manifest itself in a pregnancy woman having no confidence in herself, her body, her decision making choices. She may even choose to be in an abusive relationship or refuse to avail herself of advice and information. Inability to cope - anxiety • minor- exhibits extreme fear, stress, irrational behaviour, etc. Inability to cope and anxiety regarding the pregnancy and baby. According to Kemp and Page, coping potential is the ability of the individual and family to adapt to the stress. When individuals experience stress, they may use a variety of methods to cope. With an intense perception of threat, defense mechanisms such as denial, projection, rationalization, displacement and intellectualization may occur. The prolong denial of the high-risk status of the pregnancy may result in failure to comply with therapeutic regimes. Anxiety regarding the pregnancy and baby may manifest itself in many expressed irrational fears and distortions. According to Kuppel and Drukker, women who are having difficulty accepting pregnancy and developing a relationship with the growing fetus may present with extreme anxiety about the condition of the baby and will be hypervigilant in looking for signs that "something is wrong" with the pregnancy. Unrealistic exp ectations • minor - lack of awareness of normal requirements for pregnancy and birth Unrealistic expectations of roles of mother and or father, baby and significant others can lead to frustration, stress, neglect and abuse. Another psychosocial maladaptation of pregnancy is failure to make adequate, concrete plans for postnatal care of the baby. The absence of family members or friends to help care for the baby or, at the other extreme, passivity and over reliance on family members are signs of difficulty in adapting to pregnancy, as is unrealistic planning or inadequate preparation for managing the baby at home. Unwanted pregnancy - denial ofpregnancy • minor - unaccepting of pregnancy Pregnant women who have an unwanted pregnancy or unplanned pregnancy and/or who deny the pregnancy can tend to neglect antenatal care and leave advice unheeded. The stresses in these women are very high. Acknowledgements to: Ottawa Health Department and Ontario Ministry of Health, FORM 5070 REV88 OCTOBER NA-RISK2.CH 137 APPENDIX B T -ACE Measurement T-ACE is a measurement tool of four questions that are significant identifiers of risk drinking (i.e., alcohol intake sufficient to potentially dame the embryo/fetus). For the Pregnancy Outreach Program the T-ACE is completed at intake. The T-ACE score has a range of 0-5. The value of each answer to the four questions is totalled to determine the final T-ACE score. 1. How many drinks does it take to make you feel high? 0 1 2. Have people annoyed you by criticizing your drinking? 0 1 3. no yes Have you felt you ought to cut down on your drinking? 0 1 4. less than or equal to 2 drinks more than 2 drinks no yes Have you ever had a drink first thing in the morning to steady your nerves or to get rid of a hangover? 0 1 no yes T olerance Annoyance c ut Down E ye Opener Source: Sokol, Robert J., "Finding the Risk Drinker in Your Clinical Practice" in Alcohol and ~ Health: Proceedings of a Conference with Particular Reference to the Prevention of Alcohol-Related Birth Defects, edited by Robinson, G. and Armstrong R., Vancouver, B.C., December 1988. Note: For the purposes of the Pregnancy Outreach Program Evaluation - a client is at risk for alcohol use if she has a positive T-ACE (a score of 2 or greater). 139 APPENDIXC CLIENT DATA SHEE1 TO BE COMPLETED IN CONJUNCnON WITH INDIVIDUAL. RISK IDENTIFI.:A nON TOOL I ~ CLIENT ID NUMBER CARE NUMBER REFERRAL DATA. ; SOURCE 01' REnRRAL. 0 1•1 ALCOHOl. t. DRUG 0 COMMUNITY GROUP 0 SII.F 'j fl (t) HEALTH UNIT ~ (2) PHYSICIAN i5 I0 (3) SOCIAL SEJMCES INTAKE DATA IINTAKE ~ I I I CAn IF NO. WHY DID CLIENT NOT o 0 (7) OTHER I (51 (t) NOT HIGH RISK W&KS GESTATION ! (I) T I cue OATE ICCIMWtY) 0 ~ ~~~ -- I ovu ONO 1CLIENT BEGAN PROGIWol - ~ ~ "" __ (2) REFUSED/NOT INTERESTED :CO,.,.ENTS 0 " -- - - "" "" "" ~ - ~ ----- (3) OTHER I CLIEHTCHARA : AG£ j ~ ,...........-:-::---.....__-==--I - ; (tl MARRIED ; l'tfiST l.AHGUAGE i [ } (t) ENGLISH I c ~ IETHNIC BACXGRDUD C (2) OTHER (SPECIFY) . IF ETHNIC BACKGROUND IS 2 (NATIVE INDIAN). STATE CUEHT"S STA'I\JS 0 (t) CAUCASIAN (Nee ldMdlable) 0 (5) VIETNAMESE 0 (2) NATM! INOW. 0 (S) LATIN AMERICAN ., 0 .(3) INI)().(;ANACIAH 0 (7) OTHER U (t) STATUS INDIAN 0" RESERVE I 0 (2) STATUSINDJAHOFFAESERVE . 0 (t) GRADEl! OR LESS 0 (2) GRADU • tt ~ 1•1 NON-STAnJS : ; (5) UNKNOWN 0131Mms . 0 (4) CHINESE EDUCA110N ; i 1•1 RE\.,\T:ONSHIP (3) SINGLE (2) COMMONLAW 0 (3) COMPI..ETEO GRADE 12 0 1•1 SOME POST.SECONDARY 0 (5) UNIVERSITY GRADUAn IEUP\.OYUENT STATUS • 0 Er.IPI..OYED • OC:CVPA110N ,_ 0 (2) STUDENT 0 (3) HOMEMAKER j (t) ~ 0 --~ -------------- (t) RECEIVING INCOME ASSISTANCE (4) UNEMPI.OYEO 0 (2)\..0WIINADEOATEINCOMEBUTNOSOCIAI..ASSISTANCE 0 (3) NOTLOWINCOME ------------~--------------------------------------------------------- SPOUSE'SIPARTNER'S FINANCIAL SITUA11DN 0 0 0 ------------------------------------- 0 (t) RECEMNG INCOME ASSISTANCE T·ACESCCRE 0 1 2 (3) NOT LOW INCOME 0 (4) NOT APPLICABLE (2) LOWIINADEOATE INCOME BUT NO SOCIAL ASSISTANCE 3 4 5 CUEHT ID NUMBER: PASTPREGANCT ~ INUM8EROP PfiiEGNANCIES (Ql I I TEAM OEUVERIES I en SPONTANEOUS ASOATIONS (NS} LMHG CMII.DAEH (L) ST1UaJRTHS (S) 1DN BIRTH WEIGHT (LSW) ! AT'TEHOED PIIINATAL.C:USSU DURING A PfiEVIOUS PfiEGMAHCYT ~ L:_j (11 YU 0 !2l NO 0 j 0 (1) YES n !I (3) DON'TKNCW (3) OON"TKNCWIIJNOECICED IHAS CUEHT EVEII PIIIEVIOUSLY BEEN A POP CUEHn (2) NO 0 ~ NOT APPI.JCA8U£ CUENT MONITORING: PAE.PAEGHANCY DATE OF ASSESSMENT (00/UioiiYY) W£lGHT 80DY MASS INOEX (8lotl) I I MEAL PAT'TERN NUMBER OF MEAL$ PER CAY I NUMBER 01' SNACXS PER OA.Y , (NB: A - . . . . _ tooa.llam 310 410011 ~ • j FOOD INTAKE ( - o l - . . at-*...._ A:loar flam f ~ r~ IMNfl on Z4 ttourr.alll : GRAIN PRODUCTS ; WGETABLZS AND FRUIT i MII.X PRODUCTS I MEAT AHO AI..TERNATIVE$ FLUICS ( - o t l ! a a ( • U I ~ ,_a.yJ COFFEE fJJM:otdr$1. ~ ~ OTHER: POPS AHO SWEETENED FAUlT DRINKS (EG. KOOI.AIO, TANG) EXCLUDING FAUlT JUICES WATER KEYNUTRIEHTS(-ot.-p,_aq-onZ411outa-l IRON RICH FOODS EXCEU.ENT SOURCES OTHER SOURCES FOLATE RICH FOODS ~ OTHER SOURCES PROGRAM IHTAJCE LASTV1SIT BEFORE CEUVERY CUEHT 10 NUMBER: CUENT MONITORING.continued . ·ISUBTANCE USE PRE.PREGNANCY PROGR.UIIHTAKE I Ii CIGARETTES PER DAY DRINKS PER WEEK I DRUGS USED PER WEEK ! 1TYPE($) OF DRUGS USED (INCLUDING Ai.coHOl) ! PROGRAM CONTACT i NUMBER OF COUNSEWHG CONTACTS: ' CUEHT"S HOME PROGRAM SITC PHONECAUS NUMBER OF ATTENDANCES AT CROP-IN NUMBER OF APPOIHn.IEHTS C.UCEU.EO BY CLIENT(-.. -ltotM. no a/IGW, IIW1ottw . , _ , _ ., RECEMNQ PROGRAM FOOD SUPPI.EMENTS? IF NO. WHY NOT? Ovu ONO o SEEING A PHYSICIAN FOR PRENATAL CARE? I QYes QNO COMMENTS DATE OF FIRST PHYSICIAN CONTACT (DO/MMIVYI (APF'f!OX. IF NECESSARY! o TOTAL NUMBER OF CONTACTS WITH PHYSIC:A:>. i BY LAST VISIT BEFORE DELIVERY I REFERRALS: DURING PROGRAM I.IEHTAL HEALTH ALCOHOl. & DRUG PROGRAiooiS SOCIAl. SERVICES HEALTH UNIT PHYSICIAN NOBOOY'S PERFECT OTHER (SPECIFY) AT DISCHARGE SMOKING DATA. IGGUS 0 0 (2) STJ!ESS NUMBER ATTniPTS AT CESSATION METHODS (t) SOCIAI.ACTMT1£S 0 (Ill 0 (3) Ttt.IEOFOAY(Je.dWmMI} IARE IF YES TO SECOND HAHD SMOKE. WHEN EXPOSED? I Ovu PERSONAL GOALS (t) (5) EMOnONALFACTORS ~ 00 01'l4ER lo.IEMBERS OF YOUR HOUSEHOU) CURREHTI.Y SMOKE? 0 0 (4)110A£00M 0 ONO NOCHAHGE 0 (2) AEOOCE : YOU EVER EXPOSED TO "SECOND-HAND" SMOKE? OY!S WHERE? 0 (3) QUIT DNO ·. 0 (2) ~ 0 (3) OTHER AGE STARTED COPINCl..ntOOS HOW MANY DRINKS DOES ITTAICE YOU TO FEEL THE EFFECTS OF ALCOHOl.? HOW MANY ORINK8 CAN YOU HCU1I DRINKING PATTERNS 0 DAILY 0 P«