Empatika X PLJ LEARNING NETWORK : conducting remote research

As part of developing COVID-19 related protocols and brainstorming internally for how to adjust and adapt to the current situation, our team also had the opportunity to brainstorm with the Pulse Lab Jakarta (PLJ) team during a series of sharing sessions beginning in June of this year. We have collaborated with PLJ on previous studies including a study on Rural to Urban Migration in Indonesia. One of the things we have always liked about PLJ is that while a lot of their work is cutting-edge, they maintain an emphasis on making data work for people, sharing people’s stories and presenting data and insights in interesting and easily digestible ways.

The initial idea for the sharing sessions was to explore how to approach conducting quality qualitative research remotely, especially for more remote areas and/or with people who might have more limited access to technology or more limited experience in using it. Through these sharing sessions we hoped to develop up-to-date and contextualized approaches for doing research remotely and how to incorporate different tools and technologies in the process.

The sharing sessions were done over an initial four week period, with one session each week facilitated by PLJ. In the first session we discussed some of the possible methods that can be used for remote research that we use as a basis for the upcoming sessions. Based on some of the possible remote work we anticipated doing, we narrowed down the discussion to: Phone Interviews; Photo/Video elicitation; and Diaries.

Next we talked about what aspects we need to consider when using these methods (or any others). This included identifying the participants, the length/duration for the data collection, best practices for approaching participants, ensuring a 'safe space' for participants and other considerations such as ethics.

In the second week's session, the discussion focused on ethical considerations. Due to the different nature of remote research, such as not being able to have face-to-face interaction or do on-site observation, plus the possibility or participants sharing different kinds of personal media with us such as photos, we wanted to explore some of the different ethical challenges we need to take in account. For the discussion, we used 4 personas developed by PLJ as a way to frame discussion around the sub-topics of health, privacy, equality, trust, transparency, and respect.

Our teams note some of the technology considerations for using a diary tool with participants

The next week, we dove into the three methods to try and see what kinds of challenges we may face during the research process, particularly when working with people in more rural areas as we often do for our studies. Some of the challenges we highlighted included:

• participants' with limited technology familiarity
• bad/inconsistent phone signal
• not being able to see non-verbal gestures and cues
• ensuring participants feel comfortable (also relates to consent)
• participants availability
• participant retention over the study duration
• the possibility to misinterpret visuals (send by the researcher or participant), along with others.

After identifying these challenges, we discussed possible ways that we can tackle them if we face them during research, and any ways that we could avoid them with better prep or strategies.

The final of the first four sessions we framed around designing a remote study (using the same three methods) looking at Covid-19 impacts and social assistance. For this we spent time identifying some of the possible research questions, discussing what are the insights we are looking for and trying to contextualize the methods. For contextualizing the methods, we discussed technology (what kind of media or platform can be appropriately used, network coverage, etc.); personality (how to ensure participants to feel safe, what kind of communication approach, etc), and time (consent, when is the best time to contact, how to arrange the schedule, etc) considerations. For each item we tried to emphasize putting participants first.

All in all, these sessions were an exciting learning experience. Certainly from our end, our team left the sessions inspired and they helped broaden our understanding of the possibilities and considerations for remote research. And like any research methods, it will be a continuous learning process!

After these sharing sessions, each organization was also able to test out many of the things we discussed during our own respective remote research studies. For Empatika, we officially started our Covid-19 Remote Qualitative Insights Gathering study for UNICEF in July and are currently working on the second of three rounds in this longitudinal remote study (learn more below or click here). For PLJ, they spent much of October and early November working on a remote mixed methods study with UN Women and Gojek on ‘Leveraging Digitalization to Cope with COVID-19: An Indonesian case study on women-owned micro and small businesses’ where they worked with small/micro business owners in Jakarta and West Java (see their case study report here).

In mid-November, our teams then came together again for a follow-up session where we were able to share some of our experiences with remote research so far. We talked about what surprised each of us so far with carrying out remote research, what were the biggest challenges we’ve faced, and what were our lessons learned.

From the PLJ side, one of the pleasant surprises was that participants (small/micro business owners) were both able and quite willing to allocate large blocks of time to talk to PLJ researchers over the phone. Given the limited timeframe for the study they were working on, one of the biggest challenges was the struggle to keep their energy up and be able to juggle between conducting interviews and completing interview notes at the same time.

On our follow-up call in November, our teams share their biggest lessons learned so far in carrying out remote research

PLJ also found that from the client side, it seemed that there was a greater expectation that the research could be conducted much quicker than a regular study since interactions were just done remotely (and our Empatika team would agree that remote interactions does not necessarily equal faster/easier data collection).

From the Empatika side, our team found that carrying out research remotely is much more time consuming and draining (physically and mentally) than most of us anticipated. Some of the more specific challenges included:

• Inability to read gestures, to easily triangulate information with others, and to more easily have others involved in conversations
• For many it was difficult to match calls and WhatsApp/other interactions with participants’ variable routines and this required researchers to allocate chunks of time every day during the most intensive part of the interactions (each round done primarily over two weeks). This was especially true as many researchers are contacting multiple families.
• It was more time consuming than anticipated to explain some of the multimedia tools that we designed to participants
• And of course all of this was influenced by the fact that most of our participants are living in rural areas with inconsistent phone/internet signal.

In the end, we found that we’ve learned a lot through carrying out remote research and it has shown us that there are even more possibilities for the tools/methods available to us, and that these possibilities could also include mixing both remote and direct interactions with participants. Still, it’s clear that we’re still very much a team that likes to be in the field and interacting with people face-to-face. Hopefully the end of this pandemic will be in sight soon!