Among CFOs, the perception of the boss' importance is even greater. Fifty-five percent say a vacancy in senior management results in "somewhat" of a reduction in productivity, while 15 percent say the impact is significant.

Those who believe that the boss' departure makes a difference believe so for a variety of reasons.

Among workers who say productivity declines when a senior manager isn't present, 42 percent say the impact is a result of decreased employee morale. Roughly a quarter say it's due to increased workload (which could also hurt morale) and another 23 percent say the biggest problem is workers' concerns about what their relationship will be with the new boss and whether their jobs may be at risk.

Related: Great bosses generate good buzz among ex-employees

Only 7 percent of workers say the biggest problem created by a boss leaving is competition among remaining workers to fill the vacated position.

The CFOs surveyed also ranked worker morale and increased workloads as the top problems caused by a gap in leadership, although they were twice as likely (16 percent) to cite competition among workers as a significant problem. Alas, perhaps executives believe workers are hungrier to climb the corporate ladder than they actually are.

While the survey results display a wide range of opinions on the impact management vacancies have on organizations, there is a relatively clear way employers should interpret the data, suggests Robert Half Senior Executive Director Paul McDonald in a statement accompanying the study's release: "If a senior manager leaves a company, the focus should be on keeping teams motivated and engaged while quickly staffing the opening. Employers who don't address workers' concerns risk a decrease in employee performance and an increase in turnover."  

Related: Toxic bosses can be hazardous to your health