The Expert Panel: Enhancing Investment Decision-Making Through Collective Wisdom

In investment management, the influence of human behavior on decision-making is a critical factor often overlooked. Amid the complexity of algorithms, numbers, and data models lies an important undercurrent: human bias. This bias, coupled with group dynamics and diversity or non-diversity of thought, can significantly impact the effectiveness of investment decisions.

This paper presents a small case study that explores an innovative approach to mitigating these human influences. Drawing from diverse perspectives within Man Numeric, we show the benefits of establishing an “Expert Panel”. This panel, structured around the principles of the Delphi Model technique, brings together a diverse group of individuals to evaluate new quantitative research proposals.

The insights derived from our experiment may offer a valuable inspiration to other investment functions and managers to harness the potential of collective wisdom in enhancing decision-making processes.

Motivation

In many quantitative investment firms like ours overarching methodologies are deeply anchored in analytical research, forming the cornerstone of the business, and cultivating an environment built off academic curiosity.

This focus on innovation should encourage firms to scrutinise their operating models, specifically regarding how human behavior impacts the effectiveness of the decisionmaking processes.

They should ask essential questions such as: What influences people’s decisions? Are all voices, particularly the quieter ones, being heard in decision-making discussions? Is there enough diversity of thought to make the most effective choices?

We hypothesised that there are unconscious factors, hidden in day-to-day decisions, which may feed implicit biases. Identifying and addressing these biases could pave the way for a more inclusive and collaborative workforce.

The only way to validate our hypothesis was by putting it to the test and conducting a live experiment to assess results…and with that the “Expert Panel” was born. This study offers a wealth of insights that can be applied across a range of investment functions, with insights demonstrating the potential of collective wisdom in enhancing decision-making processes.

How to set up an Expert Panel

For this case study, we specifically looked at the process by which Investment Committees assess new quantitative research proposals. Traditionally, many committees comprise a number of individuals with a high level of expertise that have the authority to make binding “approve” or “deny” decisions towards project productionisation.

They usually cover a set number of proposals garnering supporting perspectives from outside teams as needed; however, final decisions always remain with the committee.

Companies can challenge themselves to step outside existing structural bounds by developing a new governing body of decision makers - the “Expert Panel.”

The panels are made up of a bigger group of individuals, spanning a diverse array of teams across the firm. This Panel can then act as an advisory council to the Investment Committee, providing a binary “approve” or “deny” recommendation for each proposal, along with supporting academic commentary.

This model is broadly inspired by the Delphi Model technique, established back in the 1950s by the RAND Corporation, originally as an interactive forecasting method within the military sector. It relied on a panel of experts for structured communication and decision making, all while leveraging the power of anonymity to eliminate certain biases commonly present among groups of likeminded individuals.¹

Figure 1. The Delphi Model Process Flow¹

Source: Man Numeric.

To help embed the new Delphi Model adjacent process into an existing corporate setting, one approach is to encourage researchers to present their proposals live to the entire body of experts. Following the discussion, each panelist would independently vote and provide commentary via an interactive survey.

To ensure the integrity of the process, it’s beneficial to assign an independent and unbiased third party to handle the raw, individual responses. This individual, acting as an ethical gatekeeper, would collect and anonymise the comments before distributing them back to the panel for review, thereby fostering a transparent and unbiased decisionmaking environment.

The underlying intention of this overall process change centered around the desire to broaden spheres of influence, by allowing panelists to read opinions from contrasting viewpoints, potentially opening their eyes to different angles, and sparking new personal thought processes.

In cases where a majority consensus was not achieved (benchmarked at 80%), a second round of voting would occur, providing panelists the opportunity to change their votes, if so desired. Through this, we found that shifts in perspectives were typically always attributed to consumption of the supporting commentary from others. The collective wisdom in the crowd proved to be more valuable than the wisdom found in one alone.

Figure 2. The Expert Panel Process – Wisdom in the Crowd

Source: Man Numeric.

Interesting findings along the way

Post implementation it’s important to circle back to the original questions:

Are we empowering everyone, so that those with opinions in the minority feel comfortable sharing?

As a result of this experiment, we found that individuals in the same room inevitably fell into the Groupthink² phenomenon, due to a broad-based underlying desire for harmony. This can be attributed to the fact that people typically feel compelled to prioritise loyalty to the public opinions expressed, rather than advocate for their own contrasting views in the cases where they are in the minority. It’s easier to conform and go along than to be the odd one out. In these scenarios, the desire to fit in, even if done subconsciously, ends up superseding the appetite for making the best choice possible.

This can be combatted by structuring the proposal presentations to be largely discussion based, and asking panelists to hold expression of their own personal opinions for the individual survey, as opposed to voicing them aloud in front of the group. This can help to reduce the effects of Groupthink, because, by changing the public dynamic and minimising the level of decisiveness required in front of the group, panelists are more likely to stay true to their personal thoughts and voice them accordingly.

Are we hearing from all voices regardless of seniority, expertise, etc.?

Were we hearing from everyone, or only the loud opinions from a small subset since the original method of collecting information inadvertently filters voices out?

Anonymity of the voting results helped to combat the effects of Authority Bias³ where group members tend to attribute greater accuracy to the opinion of a senior authority figure. Removing the names and titles attached to the votes, alongside discouraging individual discussions with other panelists outside of the presentations, levels the playing field by giving equal weight to all commentary.

Responses to the experiment

After evaluating the live experiment with a retrospective lens, we found we had uncovered and addressed the implicit biases that were in play. However, there was a lack of more supporting evidence to confirm whether this was truly the best and most effective operating model for teams. We turned to the panelists themselves to solicit additional qualitative feedback and the returning results were quite promising.

When surveying the experts, the majority of panelists felt an increased sense of ownership over the ideas and projects being discussed, ultimately helping lead to higher quality outcomes firmwide. The intangible boundaries between teams had been broken down, facilitating increased knowledge share and information flow among colleagues, resulting in tighter integrations between different areas of the business.

Participation levels, in terms of involvement in academic discussion and strategic brainstorming, had drastically increased, leading to greater levels of overall ‘buy in’ and autonomy across firmwide decision making.

100% of respondents remarked that the increased involvement and exposure to the proposals had been a benefit, not only to them, but more critically, it was advantageous to the business overall. In the same vein, 80% of respondents felt as if the diversity in thoughts and opinions shared had increased with implementation of the Expert Panel, compared with the original process. By expanding the pool of perspectives taken into account, both in terms of size and diversity in backgrounds and experience, we effectively reduced our blind spots by addressing them head on.

That said however, the process required a few enhancements along the way. With input from the panelists, it proved beneficial to more narrowly define the scope of proposals brought forth to the group in order to prevent burnout on the part of participants. Striking the right balance between the overarching value of time and the quantity of presentations/ voting requests circulated proved to be critical to the success of the project. Presentation moderators were also introduced as a process improvement to ensure meetings remained on track, preventing one individual from monopolising the discussion.

Results/Business Impact

Aside from the positive sentiments of those involved, what direct benefits to the investment process does the introduction of an “Expert Panel” bring? We found that algorithm refinement, incorporating complex machine learning techniques, and embracement of killing projects where appropriate, are among the top contenders.

The Panel has taken proposals for new algorithm buildouts, and through two rounds of voting, and challenging feedback, they’ve been able been to translate complexity into clear next steps of model refinement, improving overall production timelines. Quantifying the efficiencies brought forth by the Panel implementation, we’ve noted the time spent in the investment decision making phase has decreased by 35%, when looking at pre-2022 data, compared to post.

Methodologies for complex machine learning techniques were shaped through the commentary as well. From the onset, voting results highlighted a lack of consensus among the experts, but in turn, this spearheaded multiple rounds of trial and error and continuous iteration on the original research, until reaching the point of investment approval, which may never have come to the forefront without the Expert Panel.

On the contrary, having representation from Technology and Risk provides the Panel with increased insight into tradeoffs, potential barriers, costs, etc., much earlier in the process, helping strengthen the conviction behind decisions to kill projects, or to pivot directions, if it is evident that alternatives could provide similar value at lower implementation costs.

A historical comparison of the data yields materials improvements in productionisation timelines, two years pre Expert Panel vs the two years post; with the latter averaging a 40% decrease overall in the number of days spent implementing a project.

Summary

What did we learn from this experiment? By taking the time to better understand how human behavior plays a role in our day-to-day mechanics, the more we can take directed steps toward cultivating an inclusive workforce. Implicit biases, while they may live below the conscious surface, come out at the forefront of our business decisions, and by addressing them head on, we can improve our operational effectiveness, interpersonal team management, and overall workplace culture.

Pausing to ensure you have constructed an effective foundation to support business decision making, via the right mix, and the right number of individuals in the room, is advantageous in ensuring maximum value is achieved. In our case, we learned that the wisdom in the crowd is more valuable to us than the wisdom of just one or a few experts.

What started out as a short-term live experiment transformed far beyond our original expectations. It has been over two years now since we first implemented the Expert Panel, and with a newfound sense of clarity around the original hypotheses, and an improved approach to investment decision making, we have never looked back.

1. www.blog.mesydel.com/what-is-the-delphi-method-and-what-is-it-used-for-feb2d26f917a
2. www.thedecisionlab.com/biases/authority-bias
3. www.thedecisionlab.com/biases/authority-bias