To make matters worse, 37 percent said they had borrowed from their retirement accounts--38 percent of whom needed the cash for a financial emergency, and 19 percent who put the money toward buying a home.

Thirty percent overall have taken an early withdrawal from a retirement plan, while 43 percent of those who have changed jobs elected to take a lump-sum distribution rather than roll the money over into another plan or just leave it where it was.

Read: 10 least money-smart states

And this is from people who average a decent amount of income; according to Natixis, the estimated average annual income of survey respondents is $100,118.

It’s not that they don’t know they’ll need more than they’re saving.

On average, they said they will need $805,000 to fund their retirement; they expect to live on that total for another 23 years after they stop working.

However, they haven’t come anywhere near close; so far, all they’ve managed to save in their employer-provided plan is an average of $83,000.

Thinking they have outside savings? Think again: overall they only have an total of $95,000, including all sources of retirement savings. That’s just 11 percent of what they say they’ll need.

So why are they pulling money out of plans and just not saving in the first place?

There are several reasons: student debt (23 percent say they have to focus on paying off college loans) and personal debt (34 percent overall, but 40 percent of GenXers say they have too much personal debt to be able to think about putting that money toward retirement instead).

They also have some beefs with their retirement plans: 50 percent say their employer doesn’t match workers’ contributions, or the company match is too small—and they find those reasons sufficient cause to abstain from the plan.