Even if they shouldn't be planning expensive trips to take during the unlimited vacation that some companies are offering as a means of attracting and retaining top talent, that doesn't mean that people shouldn't be taking those paid vacation days and using them for some much-needed R&R.

According to Project: Time Off the 16.8 days that workers use on average is still way below the long-term average of 20.3 days that reigned between 1976 and 2000, and it's also considerably below the 22.6 days workers say they are actually entitled to. In addition, in a Glassdoor survey, U.S. workers on average respond that they have taken just 54 percent of their allotted time.

"Americans are really bad about taking their hard-earned vacation or paid time off," Scott Dobroski, Glassdoor's community expert, is quoted in the report. "I say it that way because there is a monetary value here. It's part of people's total compensation package."

Considering all the stress U.S. workers are currently under, foregoing all that vacation time could add up to a recipe for disaster—although, of course, spending money they don't have on costly vacations could be equally disastrous by boosting financial stress.

The one segment of the population that may not be in danger from leaving paid vacation time on the table is millennial women, who aren't taking vacation time in droves. Although all women took less vacation time than men across all age groups, the problem is particularly pronounced among millennials.

While 51 percent of young men say they used all their vacation days, compared with 44 percent last year, just 44 percent of millennial women, down from 46 percent last year, say they've taken full advantage of paid vacation time. And this is despite the fact that more women than men ranked vacation as "extremely" important to them in the survey.

Why on earth would they do that, you may ask. Young women are more likely to say they felt guilty, replaceable or wanted to "show complete dedication." Whether because of the fear of returning to too much work or worrying that no one else can do their jobs, more young women were concerned about the effect of vacation than young men.

Women say they are under more stress than men at home (48 percent, compared with 40 percent), as well as at work (74 percent to 67 percent). They are also more likely to say that guilt (25 percent, compared with 20 percent) and the prospect of returning to piles of work (46 percent, compared with 40 percent) impede their vacation plans.

And then there's commitment, since women also worry more than men about vacation making them seem less committed to their job (28 percent, compared with 25 percent). According to a Huffington Post report, those young women are succumbing to "work martyrdom" rather than caring for themselves in a reasonable manner. And it could be detrimental not just to their health but also to those jobs they're so grimly hanging onto.

"We need to put to rest the fallacy that 'work ethic' is equivalent to work martyrdom," Cait Debaun, director of communications for Project: Time Off, is quoted saying. "Not only are employees not getting ahead by sacrificing time off, these habits may also be harming their careers."

And Katie Denis, senior director of Project: Time Off and author of the report, is quoted in a Fortune report saying that this gap could be due to millennial men feeling more confident and secure in their jobs. She also says that millennial women are more likely to hide their need for a vacation but when they finally give up and take off, end up apologizing for doing so.

Considering the health benefits of getting away from the office—reduced risk of heart disease, lower stress levels and better concentration and focus upon a worker's return—scorning vacation time is one trend that needs to be reversed. Otherwise at least some of those workers won't live long enough to get to retirement.