Southern Ethiopia - Gwent Health Link

Field visit March 2008 – collection of evaluation data/evidence

In each of the three Health Centres data collection activities largely followed the same format although varied in ‘where and when’, depending on availability of staff, the activities of members of the visiting Gwent team and the working day of the Health Centre. The member of the Gwent team with responsibility for collecting much of the evidence did so as planned (see previous section), recording findings in copious field notes and through photographs. Others participated in specific aspects of the evaluation according to their expertise and/or remit, for example, conducting the computer skills tests and checking the computer logs was undertaken by the team member whose role was to train Health Centre staff in use of the computer and e-learning. A colleague with expertise in midwifery assessed staff in their understanding of how to manage obstetric emergencies and their ability to teach these skills to colleagues. This assessment followed prior training sessions in the management of obstetric emergencies. Observation of laboratory work and use of laboratory equipment was undertaken by a Gwent team member with expertise in laboratory practice.

Overall, data collection activities included:

Recording statistical data kept by each Health Centre, particularly statistics relating to ‘top 10 diseases’, delivery and maternal and child health. These statistics were usually recorded on hand written wall charts and displayed in the Health Centre ‘office’.

One team member making informal, unaccompanied visits to all areas of each Health Centre. During these visits photographs and notes of observations were made relating to the cleanliness (or not) of an area, availability and state of equipment, general layout and impression. In the delivery room records were examined for evidence of the type of delivery practice, for example, the number of women who had an episiotomy to assist birth. If patients were being treated in a particular area permission was sought to stay and observe or a return visit made when the room was empty. These ‘tours’ sometimes gave an excellent opportunity to meet staff and to observe their interaction with patients. Permission was always requested before taking photographs of patients and staff. The advantage of a digital camera however is immediate feedback to the subject/s of the photograph and this always led to much hilarity and many more requests for photographs to be taken. As the photographs were to be used to record detail and progress and to promote the Health Centres in a positive way, rather than be used for any commercial gain, a simple verbal request and reply was felt to be appropriate. The community and staff were familiar with the Link and work it was doing and the photograph sessions provided a record of this, of which they were very much a part. Copies of photographs will be given at return visits.

Making ‘formal’ visits to all areas of the Health Centre accompanied by Health Centre staff. Three members of the Gwent team usually undertook these reviews and used them as opportunities to teach and provide feedback as well as to assess the state and use of equipment provided in November and to assess the cleanliness and functionality of an area. The ‘formal’ tours also provided staff in each area an opportunity to convey their individual needs for equipment and resources and to feel involved. Field notes and photographs were taken to record progress (or not) and to document any teaching/advice given.

Meeting with the cleaners and environment officer/sanitarian and touring the Health Centre with them to review the cleanliness of each area. Again, these meetings /tours enabled further observation and recording of the cleanliness of each area and an assessment of the amount of use of the cleaning materials previously supplied. These were good opportunities for teaching, praising and involving the cleaners in the development of the Health Centre and were conducted by one member of the Gwent team with a second member recording observations and advice given.

Recording the equipment provided for each Health Centre and any advice given regarding its use. This will form a baseline for assessment of correct use at the next visit. A demonstration of how to use each piece of equipment was given to a group of staff at the ‘handover’ session.

One team member undertaking simple opportunistic interviews/discussions with groups of patients waiting in outpatient clinics as follows: Health Centre 1 – one group of six women waiting in medical outpatients; one group of ten women waiting to be seen in the antenatal clinic. Health Centre 2 – one group of five men and three women waiting for the general outpatients clinic, plus onlookers; six women waiting with their babies for the child health clinic, plus onlookers; Health Centre 3 – seven men waiting for the outpatients clinic. Translation was provided by members of the Ethiopian team or staff members from the Health Centre.

Undertaking opportunistic interviews/discussions with staff as they were available and as encountered when making informal tours of the Health Centre. Discussion in English was usually satisfactory with some staff translating for others if needed.

Interviewing student nurses and student midwives was anticipated. Two of the Health Centres (2 and 3) are currently used as clinical placements for students but there were no students available at the time of the visit. Health Centre 1 has no student accommodation (and therefore no students) although there are plans to build. An opportunity arose however in Health Centre 2 for a member of the Gwent team to meet with two newly qualified clinical nurses, one of whom had experience of a placement in Health Centre 2 as a student. In Health Centre 3 an opportunistic meeting was held with a group of four laboratory technician students to discuss their experiences of Health Centre 3 as a clinical placement.

Recording the discussions and decisions that took place at planned meetings, for example, the meetings between Health Centre staff and the Gwent team and the joint meetings between the Gwent team, Health Centre staff, members of the local community and the Woreda Health Office.

Observing and recording the ‘daily life’ of the Health Centre

In Health Centre 1, interviewing a representative of the women’s group.

Reviewing implementation of ‘teaching logs’ introduced at the midwifery skills workshop held in November 2007

Reviewing the ‘computer logs’ set up on installation of the computers in November 2007

Undertaking computer skills testing with Health Centre staff

Opportunistic testing of clinical knowledge and skills through small group and individual teaching and observation

Opportunistic testing of correct use of equipment through observation and discussion

The views of three Ethiopian colleagues, key to the Link, were obtained through informal discussion/interview and post visit feedback.

Providing evidence for change and impact is difficult with ‘one off’ measures. Although the Gwent team will continue to evaluate the work of the Link at each visit to Ethiopia, Ethiopian partners have been invited to contribute to the evaluation through ‘routine’ monitoring and collection of data/evidence. In each Health Centre one person was identified and invited to monitor given criteria on a regular basis (see guideline in appendix 6) and make a written report. Members of the Gwent team spent time discussing the importance of self monitoring and evaluation and reviewing the criteria with each person contracted to collect data. Apart from strengthening the evaluation process, it is hoped that participation in this way will foster for all concerned, a clear sense of ownership of change and partnership in making it happen.

Simple criteria have been drawn up for assessment at the next visit, based on advice, teaching and findings in March 2008. Together with the regular self assessment exercise, a cycle of ‘evaluation-review and reflection-action planning’ will be established.