Home » Deep Decisions – Improving Complex Decision-Making

Want yet more email?

Enter your email address to get notifications of new posts.

Request contact
If you need to get in touch
Enter your email and click the button

Deep Decisions – Improving Complex Decision-Making


Deep Decisions – An Infrastructure for
Improving Complex Decision-Making

Invitation to Express Interest

To Express Interest please use the CONTACT FORM.

This document invites expressions of interest in a project to build an infrastructure for decision-making about complex issues with long-term high social impacts. It was circulated to about 50 individuals and organisations for review in April and early May 2020. Eight individuals and organisations expressed interest and have (virtually) met on several occasions to develop the project. Please feel free to enquire if the project interests you or your organisation.

The project aims to attract three main groups of participants:

  • Group 1 – Practitioners and experts in the use of decision-making techniques. Practitioners may benefit by seeing their techniques applied and comparing them alongside others.
  • Group 2 – Policy makers and experts in issues that have high social impacts. Policy makers may benefit by having a variety of techniques that may not have been used before, applied to their issue and that could offer new insights.
  • Group 3 – Stakeholder/ Participants that may be centrally or peripherally affected by an issue. Participants may benefit by seeing new possibilities and solutions emerge to problems that have been complex and difficult to address.

The project will provide a structure for the groups to participate in action research to raise the quality of complex decision-making in practice and to document and disseminate research findings. In the long-term the project will work towards the real-time construction of organisational structures and integration of knowledge to inform decision-making in relation to complex issues.

Example decision-making techniques, issues to address and the community of participants are set out in further information that is available to those expressing interest.

The proposed project will assess how multiple techniques drawn from different disciplines, can be used together within a framework that makes decision-making more open, explicit, evidence-driven, clearer, and explainable, and also addresses issues of bias, misinformation, partial interests and diversity of values.

Action research: This is more than an academic exercise. By taking real problems and involving representative stakeholders, the project will develop and disseminate skills and techniques as it progresses. Lessons may be learned about both the applications of the techniques and the social issues addressed. The project plans to build a community and infrastructure in which the techniques become better understood, more integrated, more usable and more accessible. It hopes to create best practice in social decision-making, cross-fertilise ideas and contribute to a general enhancement of critical thinking more generally. In the longer term, the knowledge from the project may be amenable to packaging and front-ending in computational systems.

The initial round of expressions of interest for the initial project participants is now closed. However, if you or your organisation uses or champions a decision-making technique, has, or are affected by an issue that might benefit from this sort of approach, or have any other interest in the project then you can still express interest by emailing rod.rivers@mac.com at any time. This will keep you in the loop for further information.


Further Information sets out in more detail the project aims, approach, principles, proposed funding and collaborative arrangements, and example decision-making tools and issues.

Timing: This invitation was prepared prior to the concerns about COVID-19. However, the situation illustrates the need for a process, structure and repository where evidence and argument can be quickly brought together in a public forum to facilitate decision-making of high social impact. If the circumstances have release some of your time, or helped provide motivation for a project like this, then this may be the right moment to participate.

AIethics.ai is a non-profit making organisation providing support, information and resources to researchers, policy makers, designers and others interested in the safety and appropriate regulation of artificial intelligence.
To Express Interest please use the CONTACT FORM.



Examples of Complex Issues

Whilst these may be large and complex issues, they are intended for illustration only. The project is primarily about developing integrated tools and techniques that could, in due course, potentially be used to address large scale issues like these.

Artificial Intelligence and Robotics: These technologies promise to disrupt many of our current systems of production, income and wealth distribution. How they will affect society over the next few decades is a complex issue with potentially high social impacts for good, or existential consequences.

Economics and Wellbeing: As societies globally become wealthier, wellbeing becomes less to do with economics and more to do with health and social factors. Should financial factors continue as the main driver of behaviour or should we have other measures of capital and wellbeing?

Heart Disease and Strokes: Heart disease and strokes are the world’s biggest killers, accounting for a combined 15.2 million deaths in 2016. These diseases have remained the leading causes of death globally in the last 15 years. https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death

Crime and Punishment: A large number of people in prison have had disadvantaged backgrounds or suffer from mental health conditions. Are there better ways of preventing crime and treating criminals?

Education: Exponential growth in scientific knowledge and technology means that much of the knowledge and many of the skills taught to school children will be out of date by the time they reach working age, if indeed there are any jobs. How can and should the educational system be re-designed?

Tobacco: Kills more than 8 million people each year. More than 7 million of those deaths are the result of direct tobacco use while around 1.2 million are the result of non-smokers being exposed to second-hand smoke. https://www.who.int/news-room/fact-sheets/detail/tobacco

Poverty and Inequality: In the UK the life expectancy gap between the most affluent and most deprived, increased in females from 6·1 years in 2001 to 7·9 years in 2016 and from 9·0 years to 9·7 years in males. https://www.thelancet.com/journals/lanpub/article/PIIS2468-2667(18)30214-7/fulltext

Fraud and Deception: In the UK in 2017 there were 3.4m incidents of fraud costing £190bn. https://www.nationalcrimeagency.gov.uk/what-we-do/crime-threats/fraud-and-economic-crime

Climate Change: According to Bloomberg the cost of not adapting to climate change will be $1.8 trillion. https://www.bloomberg.com/news/articles/2019-09-09/the-massive-cost-of-not-adapting-to-climate-change

Homelessness: In the UK, according to the latest Government figures collected in the autumn of 2019 and published in February 2020, 4,266 people are estimated to be sleeping rough on a single ‘typical’ night. https://www.homeless.org.uk/facts/homelessness-in-numbers/rough-sleeping/rough-sleeping-our-analysis

Injuries Generally: Worldwide injuries generally claimed 4.9 million lives in 2016. More than a quarter (29%) of these deaths were due to road traffic injuries. https://www.who.int/news-room/fact-sheets/detail/the-top-10-causes-of-death

Air Pollution: It is estimated that each year 400,000 people across Europe die prematurely because of air pollution. Ambient (outdoor air pollution) in both cities and rural areas was estimated to cause 4.2 million premature deaths worldwide in 2016. https://www.who.int/news-room/fact-sheets/detail/ambient-(outdoor)-air-quality-and-health

Covid-19: As at 3/5/20 there are 4.4m confirmed cases of Covid-19 worldwide and about 0.3m attributable deaths (most of whom had pre-existing conditions). https://www.arcgis.com/apps/opsdashboard/index.html#/bda7594740fd40299423467b48e9ecf6 . One crowdsourced prediction (Metaculus) estimates a total of £1.65m deaths worldwide by the end of 2020.

Road Traffic Accidents: Each year, 1.35 million people are killed on roadways around the world. In Great Britain there was a total of 160,597 casualties of all severities in reported road traffic accidents in 2018. A total of 1,784 people were killed. https://www.cdc.gov/injury/features/global-road-safety/index.html https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/834585/reported-road-casualties-annual-report-2018.pdf .

Alzheimer’s and Dementia: There were 537,097 people with a dementia diagnosis (including Alzheimer’s) in the UK as reported in May 2019 (and possibly up to double that living with dementia). https://www.dementiastatistics.org/statistics/diagnoses-in-the-uk/

Mental Health conditions affected nearly 800m people worldwide in 2017, depression and anxiety topping the list. https://ourworldindata.org/mental-health#all-charts-preview

Antibiotic Resistance is thought to lead to the deaths of 700,000 people each year and “it is predicted to cause 10 million deaths per year by 2050 if the current situation is not improved”. https://www.antibioticresearch.org.uk/about-antibiotic-resistance/

Threats to Democracy Although the world as whole is becoming more democratic over time there are still countries that operate totalitarian control or various degrees of democracy. https://yaleglobal.yale.edu/content/globalization-and-threat-democracy . According to the Democratic Audit the UK is no longer in the premier league of democracies: https://www.democraticaudit.com/2018/10/31/in-comparative-league-tables-of-liberal-democracies-the-uks-democracy-is-judged-to-be-first-division-but-not-premier-league/

Cared for children: There were about 78,000 ‘looked after children’ (including adoption) in England reported in March 2019. https://www.gov.uk/government/statistics/children-looked-after-in-england-including-adoption-2018-to-2019

Refugees: There were 70.8 million displaced people in 2019, with 25.9 million being refugees and 3.5 million seeking asylum. https://www.unhcr.org/uk/figures-at-a-glance.html

Terrorism: In 2018 there were 8,093 terrorist attacks worldwide, resulting in 32,836 deaths. https://www.statista.com/statistics/236983/terrorist-attacks-by-country/

Knife Crime: In the year ending March 2018, in the UK, there were 285 homicides (currently recorded) using a sharp instrument, including knives and broken bottles, accounting for 39% of all homicides. https://commonslibrary.parliament.uk/research-briefings/sn04304/

Domestic Abuse: The UK police recorded a total of 1,316,800 domestic abuse-related incidents and crimes1 in the year ending March 2019. https://www.ons.gov.uk/peoplepopulationandcommunity/crimeandjustice/articles/domesticabuseprevalenceandtrendsenglandandwales/yearendingmarch2019

Relative causes of death: The infographic at https://www.visualcapitalist.com/how-many-people-die-each-day/ (published May 15th 2020) may be useful to help assess the relative impacts of these issues.

Millennium Goals: In 2000 the United Nations established eight Millennium Development Goals for achievement by 2015. The final report can be accessed at: https://www.undp.org/content/undp/en/home/sdgoverview/mdg_goals.html

UN Sustainable Development Goals: The Division for Sustainable Development Goals (DSDG) in the United Nations Department of Economic and Social Affairs (UNDESA) provides analytical inputs for intergovernmental deliberations on sustainable development, including through its science-policy interface. See the goals:


What makes these issues complex?

Mathematical complexity: Outcomes of interventions seem uncontrollable and are uncertain, unpredictable and may have unanticipated consequences. Interventions are subject to feedback loops and targets or metrics can be ‘gamed’. Stakeholder Complexity: There are few or no stakeholders that have the incentive or power to fix them. Private companies see no immediate profit. Stakeholders may have different and sometimes conflicting interests. Often progress cannot be made within the length of time a government is in power or the government has no interest in solving them. Individuals and charities do not have the resources. Benefits accrue to other groups or society at large and not necessarily to those that address the issue. Organisational complexity: Issues require people working productively together from many different disciplines to make progress.

To Express Interest please use the CONTACT FORM.



Example of Tools Applied to Complex Decision Making


These tools are not listed in any particular order, mainly because they can be applied in parallel and it is not clear yet if any one order would be more useful than any other. However, they can to some extent be grouped into tools that are basic (like fact checking and visualisation), those that are theoretically based (like complex systems theory, systemic approaches), those that primarily involve or address social systems (like stakeholder analysis, positioning theory, consensus building, idea management). However, none of these distinctions are clear-cut and the intention behind the overall approach is to be diverse, inclusive, open and eclectic.


Argumentation Tools are tools for analysing and laying out the structure of arguments. A claim like ‘CO2 emissions cause climate change’ can be argued by bringing evidence that supports or refutes it. Argumentation techniques can tease apart different aspects of a claim and highlight areas of agreement and difference (e.g. between different stakeholders). Constructing a good argument is central to untangling the issues related to complex decisions. Toulmin, S. 1969. The Uses of Argument, Cambridge, England: Cambridge University Press: https://en.wikipedia.org/wiki/Argumentation_theory. Video: https://youtu.be/T_m_HnAZln0

QOC stands for Questions, Options and Criteria. All complex issues raise many questions (or decisions), each of which have many possible answers (options or resolutions). The criteria provide the basis on which answers or options are accepted or rejected. Tools and techniques can help generate good questions, create innovative options and determine relevant criteria for making choices.

Complex systems theory: A complex system is made up of many interconnected, interacting, interdependent parts each of which has some autonomy and can adapt to change. The parts are often referred to as agents (that may themselves be complex systems) and the system as a whole may have properties that emerge from the behaviour of the agents. Complex systems theory acknowledges the limitations of traditional, reductionist approaches that tend to isolate and focus on parts of a system. It accepts that parts operate in a complex network of relationships within and between systems. Behaviour emerges from these interactions that cannot be fully understood by looking at the individual parts. Theory might help establish some of the emergent properties of the overall system such as its boundary conditions and attractors. Physical, biological, social, economic and technological systems are all complex and can only be fully understood from multiple perspectives. http://discoverthought.com/Leadership/Complex_Systems_Theory.html

Systemic Approaches see issues in the context of larger systems. “They bring together theoretical, practical, and methodological approaches, relating to the study of what is recognized as too complex to be approached in a reductionist way, and which poses the problems of borders, internal and external relationships, structures, laws or emerging properties characterizing the system as such, or the problems of: mode of observation, representation, modelling or simulation of a complex totality.” https://www.afscet.asso.fr/Archives/Systemic-Approach-eng.pdf

Visualisation Tools are used to identify patterns in complex data and explain complex concepts. Animations can help explain changes over time, including factors that feedback on each causing amplification or extinction. Visual Capitalist – 7 Best Covid Resources: https://www.visualcapitalist.com/7-best-covid-19-resources/ . Visualisation tools can be used to graphically represent the arguments in debates about complex issues: https://www.kialo.com

Fact checking: Confidence in the validity of claims that are presented as facts (i.e. as if they have a high degree of certainty) is essential to unpicking complex situations. Misinformation, whether deliberate or not, can come from all sources from governments through to individuals, and can be communicated through blatant viral memes or through more subtle presumptions. Fact checking services like Fullfact https://fullfact.org/ and Logically https://www.logically.co.uk exist to identify and counter misinformation.

Consensus Building: Techniques like Delphi can be used to help predict the likelihood of future events or the value of unknown variables. In Delphi, people make initial predictions that all participants can then inspect together with their rationales. Participants then make new estimates in a series of rounds that often converge. Adler, M. and Ziglio, E. 2002. ‘Gazing in the Oracle: The Delphi Method and its application to Social Policy and Public Health’, Jessica Kingsley Publishers. Video: https://youtu.be/awEbsBnJGPc

Prediction by Consensus: Systems that have been built to facilitate prediction by consensus include Metaculus https://www.metaculus.com/ and Foretold https://www.foretold.io/ . Commercial market research tools also exist such as Consensus Point https://www.consensuspoint.com

Red Team Thinking is a systematic way of making critical and contrarian thinking part of organizational culture. The tolols/techniques are designed to challenge assumptions, expose hidden threats, identify missed opportunities, and stress-test plans and strategies to enable better decisions. https://redteamthinking.com video: https://youtu.be/OZHvunwfdzk

Inconsistency Detection is related to both Red Team Thinking and Argumentation and may involve the identification of anomalies in, for example, reasoning, an argument, a plan, or a position. It results in a feeling that things just don’t seem to add up but may take some thinking through to discover exactly what the problem is, partly because it can be the result of mistaken assumptions in interpretation rather than what is being presented. See: http://www.cogsci.ucsd.edu/~coulson/Courses/101c/Articles/johnson-laird.pdf

The Cynefin Framework is a sensemaking and decision-makingframework that helps distinguish between simple, chaotic, complicated, and complex situations. Video: https://youtu.be/N7oz366X0-8

Risk and resilience analysis can be used to distinguish between probable, possible, plausible and black swan events. Video: https://youtu.be/2Hhu0ihG3kY

Positioning Theory is a technique for establishing where individuals or stakeholders stands with respect to some set of values (or positions). Harré, R. 2015. Positioning Theory. The International Encyclopaedia of Language and Social Interaction. 1–9. Barnes, M. 2004. ‘The use of positioning theory in studying student participation in collaborative learning activities’, University of Melbourne. See: http://wellbeingandcontrol.com/?p=1221

Stakeholder Analysis can identify different interests in relation to an issue and help understand the respective values of stakeholder groups. https://www.who.int/workforcealliance/knowledge/toolkit/33.pdf

Mediation Theory: is used to identify where stakeholders agree and differ on complex issues with a view to focusing discussions on the important differences. McCorkle, S. and Reese M.J. 2014. Mediation Theory and Practice, Sage Publications Inc. Noll, D. 2010. ‘A Theory of Mediation’, Chapter 2 in ‘AAA Handbook on Mediation’, JurisNet and Conflict Analytics Lab – https://conflictanalytics.queenslaw.ca/

Machine Learning provides methods for finding patterns in data and learning to perform categorisation tasks. Categorisation underlies decision-making that often involves mapping from complex circumstances to known solutions or treatments, and can be used where appropriate data sets are readily available. https://www.hitechnectar.com/blogs/artificial-intelligence-decision-making/

Design Fictions are a way to mitigate bias in machine learning applications and help developers and engineers reflect on the implications of wider impact of AI applications in the society. https://blog.ubiquity.acm.org/design-fictions-to-mitigate-social-injustice-in-possible-futures/

Idea Management: There are several platforms to support the collection, sorting, and analysis ideas and suggestions submitted by upto hundreds of people. https://www.softwareadvice.com/uk/idea-management/ and https://mindhive.org/aboutUs

Citizen Science is a way in which large numbers of citizens can be used to collect and analyse data, usually via some kind on online plat form (e.g. zooniverse https://www.zooniverse.org/about , EU-citizen-science https://eu-citizen.science/projects

Open Science is a way in which researchers can publish research more easily and more quickly than the traditional science publishers who often charge substantial amounts for access to scientific research (much of which has been paid for out of public funds) e.g. Open Science Foundation https://osf.io or Frontiers https://www.frontiersin.org

Mathematical Modelling: There is a large range of techniques for the mathematical modelling of complex systems such as the spread of diseases, weather and climate, crime rates, and accidents. E.g. https://www.uni-bonn.de/research/research-profile/mathematics-modelling-and-simulation-of-complex-systems-1

Each tool may already be well developed and applied to problems to which it seems suited to. The gap is in applying diverse techniques to the same problem in a way that informs by taking multiple perspectives.




Example of Participant Involvement


Evidence, observations and criteria added from a wide and representative set of stakeholder participants provides an even richer approach to complex problem solving, making it more likely that unanticipated effects and consequences will be identified.

Stakeholder Interests: Stakeholders for issues tend to segment into different interest groups. Take traffic accidents for example. Road users include drivers, cyclists, and pedestrians, all of which can be divided into sub-classes with different interests. Builders and suppliers of roads have different interests, as do maintainers and regulators of road use (local authorities, the police etc.). There are also more peripheral interests such as those of environmentalists, people affected by noise and pollution, engineers wanting to lay pipes and cables and so on. Decisions in relation to roads are therefore complex and best resolved by a diverse range of representative interests that can identify consequences and criteria that might easily be overlooked when decision-making is concentrated in the hands of a few.

Decision-Making Now: At present most decisions of social significance are made by self-selected people, many of whom have a larger than average vested interest in the outcome. The project’s proposed approach is to involve a representative sample of the population in the analysis of any complex issue. By including random samples, it is more likely that potential impacts, both anticipated and unanticipated, will be identified and taken into account.

More inclusive decisions: Policy that affects many disadvantages groups in society (homeless, disabled, poor, suffering prejudice) is often made with little attempt to include these groups, who after all, are the experts in their own lives. If Grenfell Tower residents had been involved in decision-making about the building in which they lived, it is possible that some decision-making consequences would have been spotted earlier. If the nomadic groups in Africa had been consulted about CO2 emissions we might have had a different approach towards energy use and climate change. The project starts with the premise that it is unfair, and in the long-run harmful to all, to let the decision-making of the few have consequences that affect the many without, at least including their views, arguments and interests.

Random Participation: However, the project rejects an approach where stakeholders are identified and consulted. This often leads to the artificial creations of binary oppositions when what is required is informed co-construction of better decisions and plans. Rather, the approach proposed is to consult random samples of the population so that decision making is not only influenced by sophisticated tools of analysis but also representative samples of the population who come at decisions from multiple perspectives.

For an example of the project approach to decision-making, see:

Citizen Participation in Decision Making

To Express Interest please use the CONTACT FORM.