"These days as sophisticated mega-plan sponsors perform their due diligence on potential (custom) target date solutions, advice programs and DC managed account solutions, a common focus is on the expected replacement ratio at retirement. In many cases, the plan sponsor will supply the potential vender with average (or median) plan demographics and ask the vendor for the expected replacement ratio at retirement for the average (or median) plan participant. There are a number of potential pitfalls with this approach that plan fiduciaries should be aware of."

Blanchett highlights the following points to make his argument:

Currently more than 50 percent of the Dow 30 companies use custom target-date strategies, and there is more than $80 billion in 401(k) managed account platforms.

• In order to differentiate between providers in this "custom" space, plan sponsors are increasingly requesting replacement rate studies where each provider conducts an analysis to determine the potential impact/improvement of the given investment strategy on participants (i.e., determine the "retirement success" of the participants if they use that provider's/consultant's product/solution). 
• These replacement rate studies are highly dependent on the assumptions used, and since the assumptions are going to vary across studies (perhaps materially), it is virtually impossible to compare the results of various studies and form any type of reasonable conclusions.
• Morningstar determined that making relatively minor changes to a set of base assumptions could lead to a range in the median replacement rate for a plan to vary from 48 percent to 107 percent, and a range in the replacement rate for an individual to vary from 67 percent to 261 percent.
• Therefore, while the intentions of these "replacement rate" studies may be good, they are not going to help a plan sponsor make a "quality" decision among providers. 
• A better approach would be to understand the methodology used to build the respective products and then select the approach that best matches the underlying goals and objectives of the plan sponsor.

"Given the sensitivity and potential variability of replacement ratios to the vast number of assumptions used, plan fiduciaries should never compare replacement ratios calculated by different vendors," he concludes. "If plan sponsor wanted to make decisions based on a replacement ratio study, it should likely hire a single consultant to create a controlled experiment to meaningfully compare replacement ratios."