I was on a call the other day with a company that provides a targeted consumer engagement platform based on psychographic data that is used to “bucket” people into one of a few groups and then send targeted messaging to them via different communication systems. When they got to their results slide my antenna went up.
Here were two of their claims directly off of the PowerPoint slide:

• 72% of at risk employees joined a diabetes education program versus 10% nationally

• A personalized patient messaging program created an additional $70,000 in revenue per month for a 300 physician group.

So I asked a few questions:
Me ” What data source did you use to claim that 10% of employees typically enroll in a diabetes education program?”
Him “We looked at the Rand Report (The DOL) Study which showed that 21% of employees typically engaged and then…(wait for it, wait for it)… only 50% of companies have programs, so that’s how we came up with the 10%”
To say I was incredulous would be a huge understatement.
So I asked to clarify
“So, you are counting the 50% of people who have no access to a program to begin with?”
Him “Yes”
Me “that’s just not right”
Him “Well it is true on a national basis”.  (On a side note this also may or may not be true on a national basis; just because 50% of companies don’t have a program does not mean that 50% of the employees don’t, depends on the companies in each group and their employee size.)
I was finally able to get him to admit, that the statistics he used might be a problem, when he then said “The original program was done with on-site staff enrolling employees and they were getting over 50% engagement.”
So the truer statement is they went from somewhere north of 50% to 72%, not bad, but much different than from 10% to 72%.
Now I dug into the second issue, the $70,000 per month. So I asked about this,
Him “yes, they told us that the program had increased their revenue by $70,000 per month.”
Me “lets do some calculations. It’s a 300 physician group, correct?”
Him “yes”.
Me “So lets assume that each physician generates $500,000 in annual revenue, that’s $41,666 per month. Multiply that by 300 and they are generating about $12,500,000 per month. $70,000 is just over one half of one percent of $12,500,000. That’s peanut’s, now take that and divide it by 300 physicians that equals $233 per month per doctor. Or maybe 4 office visits.”
Him “but wait, that’s profit, does the slide say revenue?”
Me “Yes it does”
Him “oh I need to correct that”
Me “yep, and still, so if its ‘profit’ they generated an additional $233 per doctor? Still seems rather small.
Him “Well that’s the number they gave us.”
Word to those who are looking for vendors and being presented with marketing materials: Ask a few Questions; you never know what you may learn.  Perhaps I should refer them to the Intel Care Innovations Validation Institute.  Full disclosure, I am on the Validation Institute’s  advisory board.