Identifying & Meeting User Needs

[Aimee Silva]

One of the challenges in visualizing data in RHIS is the wide range of different users who may need different kinds of information products or interfaces.

Some of the data users we think about in RHIS, from the national to the local levels, include (but certainly aren’t limited to!):
– HIS / M&E officers
– Policymakers
– Budget analysts
– Program managers
– District health management team members
– Health facility in-charges

Each of these different users may have different user needs. This is why investing time up front in planning for any information design activity is often critical to the success of your final product.

User centered design provides a structured methodology for thinking through your audience (users), their needs and requirements, and a rapid prototyping process that moves from mock up to proof of concept to fully functional product, all of which are tested with users for feedback. By testing early iterations with users, you can be more nimble in adapting what you create.

In thinking about our users and their needs, there are two tools from design I find really helpful: creating user personas and user journey/experience maps. Personas help you define and develop stories for the key users of your information system, while journey maps help you understand how someone interacts with the information in the RHIS and helps you to identify pain points that can be addressed in the design of your product.

Who are the different data users you’re designing visualizations for in your RHIS work? And what approaches do you use to understand and respond to their needs?

[Michael Edwards]

Thank you Amanda for your discussion of the user-centered design approach, and the link to your slide-deck.
It is also important to consider what information is needed and how it should be presented for the different audiences at each level of the system, and what are each of their different needs for the use of the data for decision making. How can our data visualizations help streamline the data analysis tasks so that the efforts can be more focused on the data use in a way that decisions lead to action.
MikeE

[tlippeveld]

Dear RHINOs
I can not more agree with Mike’s comments. I am writing you from Addis Ababa where the Ministry of Health is undertaking a review of the current set of essential indicators. Such reviews are needed every 3-5 years to make sure that the information collected corresponds to the needs of the users at various levels of the health system. Visualization of data can only be useful if the data collected are relevant to the decision making needs of the users .

Another important aspect to consider when better visualizing data is the quality and interoperability of the data collected. The Federal Ministry of Health (FMOH) in Ethiopia has developed recently a web-based data visualization and analysis platform called EDAP, which brings together various databases on services provided and on resources used. But in order to make data analysis relevant, data accuracy, data completeness, and data timeliness needs to be verified and improved where gaps are identified. Data elements need to well defined in a standardized data dictionary to allow for interoperability. The FMOH and the Gates Foundation therefore have created a new project called Data Use Partnership (DUP) to further work with HMIS stakeholders on all these aspects which ultimately will lead to better use of information for decision making.

I would like to hear from colleagues in other countries how data quality is addressed in setting up data visualization platforms (or decision support systems).
Theo Lippeveld, DUP/Ethiopia

[Tiana Jaramillo]

Hi Theo, it’s the first time I hear of EDAP, could you share the web address?

I would like to share with you an example of how data quality standards are incorporated in selected hospitals in Uganda. These sites receive financial aid from USAID, so five DQ standards required by the agency need to be monitored on a regular basis. For this purpose, SOPs have been designed, and the field teams conduct regular supportive supervision visits in which selected data elements are recalculated as per their definition, figures are reviewed for consistency, e.g. consistency between summary reports and the DHIS2 reports, and timeliness is reinforced.

Ensuring quality of data is becoming a challenge, especially when donors’ information needs change constantly. An example of this are the PEPFAR HIV indicators and the disaggregates (age group, sex, key population, test results, entry points, etc.) and frequency of reporting that change every year (or even more often) leading to the creation of supplemental reporting systems and adding a considerable burden to data staff who already barely manage to fulfill different information needs and reporting requirements.

Once data has been submitted officially to DHIS2, datasets are extracted, standardized using USAID’s open data quality standards, and analyzed/shared using Tableau. Internal controls have been incorporated to ensure that indicators are calculated correctly and that they are consistent with what has been reported to the official system and the donor.

Again, meaningful data dissemination is a challenge since most systems focus on reporting and not identifying beforehand ‘what other information is needed and how it should be presented for the different audiences at each level of the system’.

[Michael Edwards]

Thank you Theo for your (as always) insightful comments.
Our most recent forum discussed the issues surrounding interoperability, and I’m sure that issues around data quality (accuraccy, completeness, and timeliness) should also be addressed again in the RHINO Forum series.
Concerning your request for how data visualization platforms(decision support systems) address quality issues, I would like to mention a couple of examples.

In Eritrea, the Decision Support System that is attached to their HMIS has Missing Reports as one of the available Indicators that can be graphed or mapped. So the number or percent of missing reports can be graphed monthly or annually, and as with any indicator, they can drill down from the national to sub-national (Regional or Zoba, Sub-Zoba, Health Facility) levels to see exactly when and where the problem exists.

Often times, systems only report the annual percentage of missing reports by district. I think that this analysis doesn’t fully expose the type of missing data quality problem that the district has. For example, one district could have 1 or 2 facilities that are totally non-compliant and don’t send any reports, and another could have numerous facilities that intermittently have a missing report. Both of these districts could show the same annual percentage of missing reports, but the intervention to resolve the problem would be different.

In the Niger SNIS (Système National d’Information Sanitaire), a special summary report on Missing Reports can be generated that gives an annual analysis of the missing reports by quarter (quarter is the reporting period). Thus the district managers and HMIS advisors can easily determine which facilities are missing reports and when they missed sending their reports.

Concerning data quality in general, in our recent MEASURE Evaluation Mid-Term All Staff meeting, it was announced that the most requested tool used by the project was the RDQA (Routine Data Quality Assessment) Tool. I think that this is a sign that routine data sources have taken on a more important role in determining health program effectiveness.
best wishes, MikeE

[tlippeveld]

Hi Tiana (and Mike),
Thanks for your advise on ensuring data quality of data viz platforms.

EDAP was developed by the FMOH with support from Zenysis. I personally do not have access the platform at this moment. I suggest that you (and others interested) contact Jonathan Stambolis who is the CEO of Zenysis. His e-mail is jonathan@zenysis.com. Alternately, I think that Mike Edwards has the website URL.
Warm regards, Theo

[Michael Edwards]

Hi Theo,
The EDAP website needs a userid and password, and mine isn’t working anymore. 🙁
MikeE

[Michael Edwards]

Until recent advance of web-based routine information systems, dissemination of our RHIS analyses and data visualizations has been a challenge. Passing around Access databases and Excel files on discs or flash drives can’t always reach the public-at-large, or even easily reach the stake-holders that should/need access to the RHIS data for their decision-making needs.
Now, with web-based systems, we have powerful tools that can solve the RHIS data dissemination problem we had in the past. Of course there is still a problem where the Internet hasn’t yet reached, and you need a certain amount of bandwidth, but definitely progress has been made in many of the countries where we are working to strengthen health information systems.

My question is, should access to the RHIS dissemination tools, like decision support systems, data visualizers and reporting systems be password protected? I can understand the need for confidentiality with patient-based systems, EMRs and eRegistries, but what about our aggregate data? Should we have RHIS data publically available so that we can reach the ultimate stake-holders, the people in the community who use the health system? Is there a level of aggregation in the system, like the District Level that data can be publically available, but other levels, like maybe the Facility Level that we may restrict to those who provide the services?
I know the Demographic and Health Survey project has dealt extensively and appropriately with the issue of confidentiality, and thus can make data publically available at the aggregate level. What should be the policy with RHIS data?
I look forward to your opinions on this issue.
thanks, MikeE

[Aimee Silva]

I’d add another challenge around the move towards web-based systems, and would love to hear from others’ experiences.

A tool like Excel is one of the standard pieces of software on most laptops, and developing different data management, analysis, and visualization tools using the program is feasible without barriers to use like having to learn code (to build a web based viz platform or use a tool like high charts) or cost (purchasing a dedicated data visualization program like Tableau).

There are many advantages to web-based systems where web access is available, particularly for real time data updates where new information is constantly coming into the system and in use cases (as we have in RHIS) where the data volume is massive and requires a more sophisticated data structure than Excel can manage.

With accessing these systems, I’d be curious what different countries’ open government policies state about making aggregate data available down to a district level. I would expect that would have some impact on how public facing or password protected those tools are.

Any experiences out there from different countries on how this works in action?

[Jennifer Duong]

The different users for our SPRING/Sahel mapping project include: (1) our periodic users are stakeholders like our funder (USAID) and the Ministry of Health in the countries we’re working in (Niger and Burkina Faso) (2) and our more frequent users working on the ground are local and international aid organizations promoting nutrition and hygiene topics to their communities (these are the groups whose activities our project map out).

We conducted qualitative interviews for this project (in Nov/Dec) asking targeted questions from all the users (the organizations were mapped in our database) and more recently, I returned to follow up conversations with the same organizations and stakeholders.

[Author]

Posts