Who to Ask: Transferability of Findings Reported in Empirical Studies of Mediation

This is a summary of a research paper presented at the ADRRN Roundtable convened at the University of the Sunshine Coast in December 2018; comments made by ADRRN colleagues have been taken into account in this summary.  The research paper reports on one component of a much larger research project in which a systematic appraisal is being conducted of a selection of articles describing empirical studies of mediation

image001

[Vektor ID 563739124/Shutterstock.com]

When I am reading an article about an empirical study of mediation effectiveness, I want to know whether I should incorporate into my mediation practice the techniques, strategies, and behaviours that are described in the article as having been effective.  In other words, how transferable are they?

When appraising the transferability of the results of an empirical study in any field of research, two key factors are taken into account: the study’s identification of its broad sample population, and its selection of study subjects from that population.  Where neither the sample population nor the selected study subjects are appropriately representative, there is a significant reduction in the external validity of the study’s results.  In this context, it is important to establish what might be a representative mediation population.

It has been said that mediation can ‘… play a role in virtually every significant area of social conflict’ (K. Kressel, The Mediation of Conflict: Context, Cognition, and Practice, in: P. T. Coleman, M. Deutsch, and E. C. Marcus (Eds), The Handbook of Conflict Resolution: Theory and Practice (3rdEdition, Jossey-Bass, USA, 2014), p 817).  This suggests that a representative mediation population is the broad diverse community, all of whom are actual and potential mediation participants.  When mediation researchers select people to participate in their studies, it can be assumed that they are choosing subjects who represent that broad diverse community. Yet analysis of the selected mediation literature suggests that this is not the case: mediation researchers rarely mention population representativeness, and appear to choose their study subjects from a very limited range of groups (or programs):

  • Mediator and non-mediator participants in some court-connected mediation programs;
  • Mediator and non-mediator participants in some structured community mediation programs;
  • Mediator and non-mediator participants in some programsdesigned for family/custody disputes; and
  • University students (where the studies are of simulated mediation).

As the list shows, subject diversity in mediation research is restricted by the research’s own limited focus on a selection of government funded services and programs, as well as services provided through public institutions.  This focus leaves other services, and participants, largely unexamined.

The lack of representative diversity in study subjects applies to the mediator as well as non-mediator participants, and it is only one of many issues claimed by mediation researchers to affect how they are able to do their work.

The issues that have been described can be categorised as follows:

  • Obstacles (preventing certain empirical research from being undertaken): the lack of access to adequate funding; ethical restrictions that prevent rigorous examination of mediation practices and thus limit what is known about mediation,potentially disadvantaging future mediation clients;
  • Impediments(making it difficult to conduct certain empirical research): the nature of the mediation process itself (including confidentiality restrictions); the increasing incidence of mediation research being conducted in law schools where there is limited social research experience and expertise; the lack of consistent research methodologies; definitional problems (e.g., the meaning of “mediation” itself, the variety of models of practice, and the various measures of mediation effectiveness); external influences on research purpose and design (such as interest groups, funders, and researcher affiliations); and reputational concerns of potential subject mediators (i.e., if they participate in a particular study, what might be reported about them?);
  • Recurrent flawsin research design have been noted to include: heavy reliance on data collection from mediator and non-mediator self-reports; and the inherent tension between funder preferences for relatively cheap/quick studies, and protection of research rigour; and,
  • Persistent gapswhere little is known about: private mediation; mediation outcomes other than settlement;individual mediator behaviours, or microskills; how mediator values and preferences influence what they say and do in mediation; systemic issues that might influence the mediation process, and what mediators say and do within it; and the lack of comparative studies (i.e., investigations of similar mediator approaches in different contexts, or of different mediator approaches in the same context).

Other potential problems that are not mentioned often in the mediation literature include:

  • How the researcher’s own preferences and experience might influence:
    • Research design,
    • Choice of data collection methodologies,
    • Method of data analysis, and
    • Study subjects’ responses;
  • The lack of gender, race, ethnic, and socio-economic differentiation in the selection of study subjects, in the collection of data from and about research subjects, and in the analysis of that same data. In addition, not enough is known about the demographic differences between mediators in any context, nor about how those differences might affect what mediators say and do, and affect the responses and behaviours of non-mediator participants.

It has been observed that, in all fields, there is pressure on academics to publish as frequently as possible, with their research ability being assessed by the numberof published items rather than by the qualityof reported studies.  In the mediation field, this issue is compounded by the relative lack of specialist mediation publications, and the lack of sufficient mediation knowledge in other publication areas where mediation researchers do publish (e.g., law journals, business journals, social science journals); the latter can result in valuable articles not being published at all, and/or their value not being recognised.  Also, it has been suggested that publishers give preference to articles that confirm mediation’s outcome effectiveness.

Any of the above issues can influence the context and setting of an empirical study of mediation, as well as the research design and its scope, the nature of the research data that is collected, the methods used to collect the data, and the focus of the data analysis.

In particular, many of the issues are likely to influence the researcher’s access to appropriately representative populations, and, ultimately, the transferability of the study findings, and their relevance to practicing mediators.  It is important for the future practice and development of mediation that some of these issues are openly acknowledged and addressed.

The ADRRN is a valuable, respectful, and friendly forum in which mediation researchers can discuss their work with their peers.  It is also a forum in which mediation researchers can consider the above issues.  For example:

  • What are the options for improving mediation researchers’ understanding about social science research methodologies?
  • How to identify realistic and creative research funding and support that enables:
    • Access to a broader and more representative population of subjects for empirical research;
    • Access to diverse mediation settings and diverse research subjects;
    • Empirical investigations that are more complex and innovative than evaluations of mediation outcomes; and
  • How to encourage the dissemination of, and access to, mediation research, without being guided solely by results and findings?
Advertisements
This entry was posted in Dispute resolution by AlysounBoyle. Bookmark the permalink.

About AlysounBoyle

Alysoun is a PhD candidate at the University of Newcastle Law School, investigating mediator effectiveness. She is a member of the ADR Research Network, ADRAC, the American Association for the Advancement of Science (AAAS), and the Law and Society Association (USA). She was a member of the American Bar Association (ABA) Section of Dispute Resolution Task Force on Research on Mediator Techniques until that Task Force's disbandment following publication of its Final Report in 2017. The Final Report included recommendations for future activity in the field of mediation research, and Alysoun continues to work with other key former Task Force members to progress those recommendations. Alysoun is a member of ADRA, and a former Director of Resolution Institute. She is a Co-Convenor of the National Mediation Conference being held in Canberra in 2019, and is Chair of the Conference Design Committee. She is an experienced DR practitioner and educator/trainer. She is also the Training Officer and Call-Out Officer for her local brigade in the NSW Rural Fire Service. Contact: alysounboyle@gmail.com

1 thought on “Who to Ask: Transferability of Findings Reported in Empirical Studies of Mediation

  1. Pingback: UKAJI January update | UKAJI

Post your comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s