How Google Used Data to Validate the Impact of Good Managers

"At Google, we strive for all people decisions to be informed by data and analytics," says Kathryn Dekas, people analytics manager. Google, which employs nearly 60,000 employees (referred to as "Googlers") across more than 100 offices in 50 different countries, has a culture that's naturally attuned to data—its founders were both engineers— "And as we all know, engineers LOVE data," says Dekas. 

From Google's beginning, Dekas says that it was understood that in order to have a seat at the table and participate in driving action, every functional area within the company absolutely had to speak the language of data, HR included.  

"Google may have in this way predated what's happening now in terms of being able to speak the language of data and convincing people using data in order to have impact. That's one element of our culture that's really shaped the HR organization." In fact, Dekas says, the key pillars of Google's culture—to not be a conventional company and to use data in daily operations to make decisions and drive action—have been key influences in how HR at Google formed and has evolved over the years. 

The people analytics group at Google is composed of a diverse group of individuals, one-third of which hold Ph.D.s in disciplines such as psychology, behavioral economics, organizational behavior, and sociology. One-third are MBAs who focused on human capital and other types of HR topics within their MBA programs, and the last group has diverse backgrounds, but all share expertise in data mining using advanced and emerging tools. The people analytics team reports directly to Laszlo Bock, Google's SVP of People Operations, and each of the other seven functions of the HR organization that also report to Bock has a dedicated people analytics team member (called points of contact) working with them: talent; benefits; sales HR; technology & operations; general & administrative (G&A) HR; staffing; and engineering and product development. Through the work of this group, all HR decisions, whether it's compensation, talent management, staffing, etc., are made with data behind them. 

The Google Analytic Value Chain, as Dekas explains it, begins with opinion (which is how some people outside of HR may believe all HR decisions are made). Next is data, which HR has always had a great deal of—how many people are employed by the organization, how many employees get promoted, how many leave the organization, etc., but while this information may be structured, it's raw and often not easy to digest. The next level up the chain is metrics—counts or ratios that HR may send around on a regular basis, e.g., the percentage of managers for each employee. Tracking trends can be helpful and illuminating, but over time, Dekas says, the sending around of dashboards of metrics can become routine, without any action taken based on them. Where things start to get interesting is the next step, which is analysis of the data—identifying relationships, trends, or special populations, and making connections. Next comes insight taken from the data analysis, which is then leveraged to drive change in the company. Insight leads to the next step, action, by influencing decision makers, who then take action in the forms of policy or process change, or new programs, for example. 

One example of using data to drive change at Google is Project Oxygen, which investigated whether managers really matter at Google, and if so, how to ensure all employees, in Dekas' words, could have "an awesome manager." 

At one point in the early days of Google, the role of people managers in the engineering-driven culture was not understood. So the decision was made to eliminate managers and everyone became an individual contributor. For many reasons Google eventually brought the managers back, but still, there was no real proof that managers brought impact or value to the organization and so the question of the need for people managers lingered (Garvin, 2013). This conundrum prompted members of Google's People & Innovation Lab, a group of social scientists within the people analytics team known internally as "The PiLab," whose mission is to focus on solving longer term problems, to take on this question—do managers really matter?—as a project. 

They began with data that already existed: performance reviews (view of the manager from a top-down approach) and employee surveys (view of the manager from the bottom-up). The two scores were plotted—one on an X axis and one on a Y axis, and the researchers looked at where the managers fell in the plot. At first they noticed that there was a fairly tight cluster and that managers seemed to be doing pretty well. But then they cut the data into the top and bottom quartiles and looked at the best and the worst managers and how their respective teams performed (team productivity, engagement levels, retention, etc.), and this is where they started to see huge differences. 

At this point, the PiLab researchers had been able to demonstrate that managers indeed had impact on things the company cared about, but they didn't want to stop there. They wanted to go further and help managers who were struggling to improve. They went with a two-prong qualitative approach: first, they created a "Great Manager Award" and solicited nominations that required that those nominating managers write up examples of behaviors the managers displayed that the nominating person believed made their manager a truly great manager. The researchers studied these nomination write-ups and coded them to identify common examples of behaviors that great managers displayed. 

Simultaneously, the researchers conducted a double-blind interview study of a pool of managers (neither the researchers or the managers knew which group they fell into—good manager or struggling manager), and the resulting transcripts of these interviews were coded as well to track and better understand the commonalties among the good managers and what they had to say about how they did their jobs. Based on these two studies, the PiLab came up with behaviors that Google's best managers consistently exhibited and three behaviors that struggling managers consistently exhibited. 

The research found that a high-scoring manager at Google is: 

• A good coach
  
• Empowers the team and does not micromanage
  
• Expresses interest/concern for team members' success and personal well-being
  
• Is productive and results-oriented
  
• Is a good communicator—listens and shares information
  
• Helps with career development
  
• Has a clear vision/strategy for the team
  
• Has important technical skills that help him/her advise the team  
  

The most common pitfalls among managers who were struggling were: 

• Has had a tough transition (e.g., suddenly promoted, hired from the outside with little training)
  
• Lacks a consistent philosophy/approach to performance management and career development
  
• Spends too little time on managing and communicating 
  

These obviously aren't particularly revelatory findings about what makes a good manager, but the point is that if a group of leaders were polled to come up with ideal behaviors of effective leaders, chances are the group would quickly come up with a list of 20 to 30 behaviors. Google knows for certain that these eight aforementioned behaviors are the ones that top- performing managers in their company exhibit, and that is truly powerful insight for them. 

"This is data that we can do something with. We can start measuring against it. So one of the things we did was start twice- yearly upward feedback surveys. Everyone's direct reports rate their managers and the managers then get the results and are held to an expectation that they will take action to improve," says Dekas. 

The Great Manager Award is still in place as a way to recognize managers who excel in their roles, and based on the Project Oxygen study, manager training has been revamped and new communication plans have been created to let people know what they are being measured on and what the best practices are that managers at Google do to improve their performance in specific areas. 

Looking back at the Google Analytic Value Chain in terms of the Project Oxygen example, we see that they began with opinion: "Managers don't impact Googlers' performance—they don't really matter." They had data: they knew how many managers they had, but knowing that number wasn't a reason to keep all of those people managers in their roles. They had metrics: they knew that most Googlers had favorably rated their managers. Then they went into the analysis phase, which is where they discovered that good managers—their best managers—had statistically significant impact on things that the company cares about—they have teams with very low turnover and higher performance than those of the company's struggling/worst managers. From this analysis they gleaned insight and put together a list of eight behaviors that great managers at Google exhibit, which provides a roadmap toward action on what to train people managers toward in the future to ensure continuous improvement.

This case study is an excerpt from the i4cp report The Promising State of Human Capital Analytics