Data from the U.S. Bureau of Labor and Statistics shows that 11.3 percent of men age 25-54 are not working, and more than four in five of those have not worked in the past year. In 1954, only 2.1 percent of men in that age range weren't working.

"One in six prime-age guys has no job," Nicholas Eberstadt, a demography researcher at the conservative American Enterprise Institute, told NPR. "It's kind of worse than it was in the depression in 1940."

Related: Jobs that are hot, and jobs that are not

The same research shows that, unlike women, men leaving the workforce are rarely doing so to take care of kids. While 40 percent of women who aren't working are primary caregivers, that is the case for only 5 percent of nonworking men.

While some, particularly on the right, are quick to argue that safety-net programs — such as long-term unemployment benefits — encourage people to forgo work, that argument is harder to make when the labor participation rate is higher in western countries that provide far more generous welfare programs than the United States A recent study by the White House attributed only a small portion of nonworking men to Social Security disability insurance, for instance.

Indeed, some argue the problem is a lack of government initiative to help out-of-work people get training and find jobs.

Related: C-suite remains male-dominated

One major difference between the United States and other western countries that can almost certainly explain part of the problem is its enormous prison population. Roughly 3 million Americans — overwhelmingly men — are in prison, and millions of others have criminal records that make it difficult to find employment.