19 Jan Evidence-based HR: Under the microscope
By Katie Jacobs – hrmagazine.co.uk
The evidence-based management movement has HR in its sight. But is the profession willing or able to become more evidence-based?
Eric Barends was working on a change programme in a large organisation when he first got the feeling something wasn’t quite right. “Three directors were appointed to manage the change and they all had completely different views on how it should be managed,” he recalls. “I thought, ‘how can it be possible to have three different approaches to solving the problem of transition?’ Imagine being in hospital and having three doctors, all with different ideas of the best treatment.”
Intrigued to learn more about the decision-making process, Barends asked the three where they were getting their ideas. “Nine times out of 10, it was down to personal experience or one survey they’d read.” Perturbed by the lack of substantial evidence, Barends started his own research into critical thinking and decision making. He eventually came across the concept of evidence-based medicine.
Evidence-based medicine emphasises the use of evidence from robust research in healthcare decision making. Originally coined in the 1960s, the prefix ‘evidence-based’ has detached itself somewhat from medicine, and can now be found attached to fields as varied as education, criminology, conservation and management. The catch-all term is ‘evidence-based practice’. Barends, who in 2011 set up the Center for Evidence-Based Management, refers to such practice as “an activist thing”. “Evidence-based practice is fighting against fallacy and fads,” he adds.
Given that definition, perhaps it’s no surprise academics are now keen to spread the practice and understanding of ‘evidence-based HR’ (EBHR). Rob Briner, professor of organisational psychology at Bath University’s School of Management, says HR is like “any area with a lot of fads – not very evidence-based and lurching from one thing to another”.
A new paradigm
Briner adds that evidence-based management is “not a wild and crazy concept”. “The term ‘evidence-based’ describes something people always do, which is basing decisions on information, but doing more of it, doing it better and doing it more critically,” he says. Indeed, as Barends points out, “it can be seen as common sense”, but, he adds, “the more you dig into it, the more you see it’s a completely new paradigm”.
Denise Rousseau, H.J. Heinz II professor of organisational behaviour and public policy at the US’s Carnegie Mellon University, explains: “The real issue in this movement is to call attention to the quality of the evidence people are using. If you ask HR practitioners if they are paying attention to the quality of their evidence, it knocks them back.”
Rousseau describes EBHR as “being concerned with using the best available evidence in making decisions pertaining to HR profession and practice”. Four equally important areas should provide input into that decision-making process, and all four should be assessed critically and systematically. They are: external academic and scientific evidence in the field of HR, good quality internal data from your own organisation (hard and soft metrics), professional expertise and experience built up over your own career, and the values of stakeholders (ethical and practical concerns – is it the right thing to do for your people?).
Consensus from academics is that working in a more evidence-based fashion leads to better quality decision making, increased credibility for the profession (or professional) in question and, in Briner’s words, could “stop the fad cycle”. “If we as HR executives tie our activities to what the evidence is showing, the evidence will make our case,” adds Rousseau. “It’s not rocket science; it’s a more strategic way of thinking about things.”
According to Wendy Hirsh, an independent researcher and professor at Kingston Business School, who authored a paper on evidence-based HR for the Corporate Research Forum in 2011: “Being evidence-based is partly about how professions see themselves.”
Back in 2011, Hirsh wrote: “The jury is out on whether EBHR will take off. We need a period of co-creation in the idea. Although academics coined the phrase, practitioners will need to make it their own before they commit much effort to thinking or behaving differently.”
Speaking three years on, Hirsh believes EBHR “is a journey, not a destination”. “This is also a mindset,” she adds. “Do HR people see themselves as professionals, in terms of critical thinking and reflective practice?”
Looking for evidence
Leading-edge practitioners would say they do. “I believe in evidence,” says Eugenio Pirri, VP HR and OD at The Dorchester Collection. “We live in an age where there is no reason not to have data and evidence around what you decide to do, or to figure out and examine your key metrics.” Pirri himself has worked with New York’s Cornell University on research projects in his own organisation, including linking employee and guest satisfaction to drive performance. “You can’t go forward unless you know what your insight is going to be and what you want to achieve,” he adds. “It’s all very well to say ‘I want X’, but why do you want it?”
This echoes one of the most critical areas for getting EBHR right: starting with the right question, which is often ‘How do we know we have a problem?’ Jeffrey Pfeffer is Thomas D. Dee II professor of organisational behaviour at the Stanford Graduate School of Business. Having studied organisations and management for decades, he feels that all too often “HR is about programmes, not about thinking”.
“There are tonnes of companies that put in programmes because they think it’s the thing to do, but they don’t ask: Do we know we have a problem?” he says. To illustrate this point, he tells an anecdote about an HR director he once spoke to from a large bank. “She called me and said: ‘My CEO wants to put in a programme to improve retention of high potentials,’” he recalls. “I asked if they had gathered any data to suggest their high potentials were leaving at a higher rate than in the past, or more than their competitors. Her response was: ‘You must not have heard me, I said my CEO wants a programme for high potentials.” Eventually Pfeffer told her to go to a big consultancy, “who would be happy to do a programme whether or not there is any need for it”.
This example perfectly illustrates the bind HR can find itself in. Paul Kearns, chair of the Institute of Maturity and former head of HR for an automotive manufacturer, believes at least some blame for a lack of evidence in HR can be laid at the feet of CEOs. “I blame CEOs for looking for off-the-shelf answers and HR for being too willing [to give those answers],” he says. “Up until now, it has been career limiting to ask too many questions of CEOs. A lot of senior HR people have based their careers on saying ‘yes’ and not challenging.”
The problem with best practice
Hirsh agrees HRDs are “put under pressure to do the things that big name firms do”. She uses the example of the “extremely problematic” nine-box grid used by many organisations in talent management. “People who know nothing about talent management know it exists, so ask why you aren’t using it,” she says. “There’s a pressure for HR to adopt things because someone else is. This naive idea of ‘best practice’ has really inhibited EBHR.”
For Kearns, “the HR community is still trying to hide behind best practice”, which means “there is a bigger need for EBHR than ever before”. Rousseau compares the proliferation of fads in HR to “voodoo”. “If you don’t understand science, why not follow voodoo?” she adds. “Saying ‘it’s what everyone else is doing’ is a way for those without the mindset to think like their peers to cover up for a lack of knowledge.”
Many of the academics HR magazine spoke to were also rather dismissive of consultancies, accusing many of peddling products based on dubious ‘best practice’ without robust evidence behind them or a significant interest in evaluating their work. “Marketing by consultants tends to have a disproportionate impact, and I think that becomes dangerous, as it can be seen as evidence,” believes David Guest, professor of organisational psychology and human resource management at Kings College London. However, consultants could argue they are not asked for evidence, with one anonymous consultant saying clients often “don’t give a monkey’s” about evidence.
This suspicion of ‘best practice’ extends to many in the practitioner community too. “I want HR to be the fact guys, not the fad guys,” says Pirri. ActionAid head of HR Graham Salisbury believes: “The Center for Evidence-Based Management asks challenging questions and as a profession we are not prepared to look at the evidence, which is quite disturbing.”
“Where we do try and adopt a more scientific approach, we aren’t fantastic at telling fact from fiction,” adds freelance OD consultant David D’Souza. He first became interested in EBHR several years ago while working in-house at an international business, exploring broader sources of evidence to figure out why HR practices weren’t having the desired impact. “Having hard science sitting behind [what we do] is a beguiling prospect, but there are issues that aren’t being as well articulated as they could be,” he adds. “The modern HR professional needs to be able to separate the fact from fiction, signal from noise and commercial opportunity from fad.”
An academic issue
When it comes to how HR can achieve what D’Souza describes, it requires a mixed bag of tricks. One of the problems with EBHR and evidence-based management is it can risk sounding as though academics are simply complaining – frustrated about the lack of attention practitioners are paying to their work. This is the impression business author David Bolchover, who is working on a report on the subject, has gained. “The thing that really struck me is this big divide between HR academics and practitioners,” he says. “HR academics who have been writing evidence-based research for decades are annoyed that HR practitioners are ignoring what they write.”
Hirsh believes one of the issues is that there is “no clear academic canon around what HR professionals need to know”, and that what exists isn’t the easiest of reads. “Areas like reward and engagement are hotly contested within the academic community,” she explains. “We sometimes know what correlates, but not what causes. Looking externally for good evidence to support practice is really problematic for practitioners. There is not that middle ground [of research] that really supports practitioners, telling them what they need to know from the research evidence, but in terms they can reasonably apply.”
Hirsh adds that she would like to see HR practitioners doing more “experiments” on their workforces. By this, she means less clinical trials, but more concepts like trialling a practice in one area of the organisation, such as in one country or site, and not in another to see how the results differ.
Salisbury bemoans the fact he is, in his experience, one of a shrinking number of HR practitioners who reads academic books and journals. “I do worry we are witnessing the death of book and journal reading habits in the HR profession,” he says. But he also believes “academics often write for other academics”. “There are not many areas where academics are writing for HR professionals,” he adds. “There’s something that needs to be done to meet in the middle.”
And while Pirri believes HR leaders need to proactively work with business schools and universities in HRM research, he calls on more academics to come into businesses. “They can theorise all they want, but they need to understand how our businesses work,” he points out.
There is an acknowledgement of this from the academic community. Guest appreciates that “we [academics] should be doing a lot more”. “We have to find ways to communicate more effectively, writing in a more accessible way,” he says. “We have to get down among the practitioners.” One of the solutions he suggests is practitioners forming focus groups to “translate” the latest academic work into a more user-friendly format, possibly with the support of professional bodies like the CIPD.
The internal data solution
For Sara Rynes, John F. Murray professor of management at the University of Iowa’s Henry B Tippie College of Business, the key to getting more practitioners interested in evidence lies in making the most of internal organisational data and metrics. “Most people are not going to read academic research, we have to find some other way of creating evidence-based management inside your company,” she says.
This requires intelligent analysis of data – not merely collecting reams of management information or buying a fancy analytics system. “Start with what the problem is,” says Rynes. “Get thinking about the stuff you want to fix. Then think about how you can use data.” But both Briner and Rousseau add that for truly EBHR, while internal data is useful, it shouldn’t be used at the expense of the other three areas.
In practice, there are naturally some areas of HR in which it is easier to start working in an evidence-based way. Academics cite selection and recruitment as one such area, due to the amount of research and data available on the subject, meaning it can be easier to make an impact quickly. Rynes also points out that as “people adversely affected by [the recruitment] process don’t become employees” it can make sense to start experimenting with evidence-based practice in this area.
However, what is required most of all is a mindset shift, the ability to think analytically, see HR data in a wider context and use it to test hypotheses. Take recruitment, for example. “We know way more about selection, but the predominant way of hiring people is still through gut reaction, and that is so not evidence-based,” sighs Rynes. Pfeffer calls out the fact that “most companies don’t even measure the success of their recruitment processes” by failing to link recruitment evidence to retention and performance.
“Internal evidence collection and use could be done way better,” says Hirsh. “People are thrashing about with analytics, and evaluation is patchy.” D’Souza points out the skill set of the HR profession has “historically not been aligned to the sifting and sorting of data and analysis to provide genuine insight”. “We need to make sure the first faltering steps we take don’t end up with us simply falling over,” he adds.
At ActionAid, although Salisbury says he is not using data analytics, evaluation is key. “We have groups of people here who work in monitoring and evaluation. If we are spending money from donors or the Department for International Development, we have to demonstrate we’ve done what we’ve said we’ve done, and that filters through to HR,” he explains. When running a recent management development programme, Salisbury’s team decided on four specific outcomes they wanted to achieve, and at the end of the course conducted a thorough face-to-face review with all those who took part, asking them to rate it against its objectives, rather than using what Salisbury dismisses as “a happy sheet” to evaluate success.
What can be harder are what Hirsh refers to as the “deeper issues of motivation, engagement and leadership”, as “here cause and effect is really difficult to unpick”. However, according to Guest there is plenty of good work being done in these areas, although it risks “being lost in academic journals”.
And Pfeffer believes that the “idea stuff can’t be measured is nuts”. “Get a group of smart people together and they will come up with a measure,” he says. “This is about discipline and engaging in disciplined process.”
Think like a scientist
It’s also, in Pfeffer’s words, about “thinking like a scientist” rather than persisting in “thinking like a clerk”.
Pirri, who has a background in operations and finance, agrees a lot of HR professionals “haven’t been taught how to get evidence and don’t have a systematic approach”. “They don’t understand the thought process and can get nervous, which is why they hide behind best practice,” he adds.
Thinking like a scientist and engaging in ‘mindful practice’ means being intellectually curious, evaluating past decisions and questioning ‘known knowns’. “If you ask ‘why’ it means you are on the way to becoming an evidence-based practitioner,” says Briner. “If you say, my boss told me, I read it somewhere, or a consultant said it, so it must be true, you are not.”
“Critical thinkers are more likely to be aware you can start with the wrong question,” says Rousseau. “You need to be able to ask good questions about why something is needed.” Hirsh believes experts should be trying to help practitioners frame better questions, rather than jumping in and asking ‘where is the evidence?’, which can naturally lead to defensiveness. “If you are going to benchmark, you need to ask better questions,” she says. “You need to start with the issues being addressed, their context and how you might address progress.”
Of course, asking questions can mean messing with the status quo and requires courage. “Evidence-based management challenges assumptions made by people in senior positions,” says Barends. “The whole thing is about accountability. Asking senior people what the evidence is for their decisions is threatening.” However, with people issues being linked, directly or indirectly, to so many organisational issues, Barends believes not using or asking for the best available evidence is more dangerous.
On a practical level, academics claim that although time is often raised as a barrier for working in this way, evidence proves this is not the case. “We have evaluated thousands of people working in this way and no-one says they use more time; they use time differently,” asserts Rousseau. Barends adds there is no need for everyone in an organisation to be an evidence-based practitioner either; it can be more effective to use a smaller group of people working in a consultant-like capacity.
Evidence-based cultures
What is critical is the culture of the organisation. “This is not just an individual thing,” says Briner. “It’s about organisations and is a cultural thing. If you ask questions, you need the support of your group, team and organisation. People can get annoyed if you say ‘it depends’, but that’s the right answer much of the time. It does depend and it’s your job to work out what it depends on.”
Kearns agrees: “The organisation needs to ask for evidence. Even if HR people want to make a difference in terms of EBHR, they can be constrained by a lack of maturity in the organisation.”
That’s why Rousseau believes senior HR practitioners need to take the lead in this field, as they have the luxury of having “autonomy”: “They are the best able to take the first step into evidence-based practice as they have control over their own practice.”
She adds that the “freebie benefits” of this type of practice come quickly: “Even if you consider one alternative, the quality of your decision goes up.”
Kearns believes it is significant that in this still emergent area HR is taking centre stage, in academic work at least. “Generally evidence in people management is problematic and that is why, interestingly and positively, the HR side of things is having to lead the whole of the evidence-based management movement.”
However, he adds that although this should be “very exciting” for HR professionals, he remains sceptical over whether many are actually interested.
Rousseau is far more positive about the potential rise of EBHR. She points out that the whole area is still less than 10 years old. “The change is looming, and we are starting to see take up,” she says. “This is still a new movement. It took a generation of physicians to create evidence-based medicine. This will take a generation of enlightened managers.”
Whether or not many of those enlightened managers will come from the HR community remains to be seen. Can HR be truly evidence-based? To echo Briner, it depends. It depends on practitioners and professional bodies moving towards the professionalisation of the function. It depends on academics working with practitioners to produce truly useful and accessible bodies of evidence. And it depends on HR enhancing capability to create insightful data analysis.
And as debates continue over the future of HR, it is perhaps worth keeping ‘Pfeffer’s Law’ (coined in his 2006 book The Knowing-Doing Gap) front of mind: “Instead of being interested in what is new, we ought to be interested in what is true.”
Subscribe for the best of HR Management direct to your inbox:
No Comments