Consumer financial adviser study ‘seriously flawed’

The Consumer study of financial advisers released last November was seriously flawed according to an independent report and the institute could end up facing legal action.

Thursday, June 10th 2010, 5:11AM 16 Comments

by Jenha White

Plan B and Rutherford Rede commissioned the report written by Michael Mintrom, an expert in research methods and public policy analysis, after both financial advisory companies had their advice rejected in the Consumer study which described financial advice in New Zealand as "scandalously poor".

Consumer says the purpose of the study was to see if the standard of financial advice had improved and it received financial support from the Ministry of Economic Development and the Retirement Commission.

Overall it mystery shopped 33 financial advisers from large institutions with in-house advisers and agents, sharebrokers and nationwide adviser chains, to small standalone firms. An expert panel assessed the quality of advice and information in the 17 plans it received.

Only three out of 17 advisers produced plans that were rated "good" by the expert panel and the remaining 14 were rated as "disappointing" or were "rejected".

According to Mintrom the Consumer study was motivated by good intentions but the quality of the study was poor, especially when judged by the number of cases involved, the high rate of attrition of cases during the study and the lack of screening and training of the mystery shopper.

Mintrom says if this study had been done through the University of Auckland, the design would not have even made it through an ethics board.

One of the main concerns is the naming and shaming approach. Mintrom says this strategy should be avoided unless there is high quality and robust evidence, which is hard to obtain.

He says the sample size of the Consumer study was one of the problems. The Institute of Financial Advisers has 1,400 members, yet the decision was to include only 33 firms in the study.

Only 29 were assessed because four plans arrived too late for assessment by the panel. Of those 29, another eight firms were dropped because of a mismatch between the shoppers need and the adviser. Of the 21 remaining, another four only gave oral advice leaving just 17 responses to be judged.

"The attrition of cases from the original sample of 33 suggests very poor project management," says Mintrom.

He also says the mismatch between the shoppers and advisers is clear evidence that the people who managed the project had no understanding of how their own incompetence could bias the results they obtained.

Mintrom suggests a study of at least 100 firms would have been needed to be more representative of the financial adviser industry and at least two to three mystery shoppers would have needed to visit the same firm to be sure of the level of advice.

"If you want to engage in naming and shaming, you have to be dead sure you are correct in the claim that a company is not giving good advice."

He says it is especially troubling that the mystery shoppers got to choose their own scenario.

It is reported that "eleven mystery shoppers ranging in age from their mid-30s to just over 80 visited financial planners... These were real people with real financial questions".

Mintrom says there is a fundamental problem here because the diversity of the mystery shoppers is very likely to have affected the quality of the advisory sessions.

"Hardly any effort was made to control the character profiles and test scenarios, as a result, the quality of the advisory sessions would have been influenced in unknown ways by the knowledge and behaviours of the mystery shoppers, which Consumer did nothing to control."

Consumer said in eight of the 33 cases, "there was a mismatch between the shopper's needs and the adviser" which Mintrom believes points to a major flaw in the study that very likely corrupted the findings.

He says studies that make effective use of mystery shoppers devote time and energy to developing realistic character profiles and test scenarios.

Mintrom believes that while Consumer had good intentions to start with, the way they went about it could end up having legal implications if companies decide to go down that route.

He believes the report is detrimental to the investment climate in New Zealand especially when the government is trying to get people to diversify with investment.

"I do have concern that taxpayer money was spent on this when the quality of the study was so poor.

"We must question why a survey of this kind, with such potential influence, was left to Consumer rather than a professional research house with the experience and background to manage the entire process effectively."

Consumer was not available for comment.

Jenha is a TPL staff reporter. jenha@tarawera.co.nz

« Brook calls lawyers into scrap with Devon movesKiwiSaver mismatch a 'huge challenge' for advisers »

Special Offers

Comments from our readers

On 10 June 2010 at 8:16 am Independent Observer said:
Congratulations on Plan B and Rutherford Rede for commissioning the report written by Michael Mintrom. It's pleasing to see someone standing up for the industry (take note IFA).

I hope that this latest scandal gets properly aired by the daily media, so that ordinary investors can see who really was the loser in this scenario.
On 10 June 2010 at 9:34 am Flite said:
Once again IFA has shown a distrubing lack of value, shouldn't they, as an industry advocate had lead this kind of thing?
On 10 June 2010 at 10:20 am David Whyte said:
I can't claim to be an expert of Michael Mintrom's standing, but I did spend almost a year as MD of Taylor & Associates in Melbourne, and I learned enough to realise that the Consumer 'survey' was a disgrace. The statements made and conclusions reached were not supportable from the data gathered - mainly due to the reasons outlined by Mr Mintrom. However, unlike Mr Mintrom, I have a less charitable view of the Consumer's motives. The funding of the initiative perhaps contains some clues. The complete lack of objectivity apparent in some of the more emotive statements made in the report, and the lack of appetite to retain a professional research facility to provide more reliable findings, point to suspect motivation. The apparent glee with which various commentators alit upon the study's findings - and the nature and substance of the interests represented by those commentators - also cast serious doubts over the integrity of the methodology deployed, and over the motivation which gave rise to this poorly constructed initiative in the first place. Incompetent at best, ill-conceived - certainly, malicious - possibly - but overall unworthy of the claims the study made to cast doubt over the quality of financial advisers operatig in NZ. Those responsible for commissioning, funding, and implementing this nonsense should be ashamed of themselves. Unfortunately, nobody is likely to be held to account for this shambles and the daily media will simply ignore the issue.
On 10 June 2010 at 11:39 am Johnny Adviser said:
Unfortunately this is true David. Had it been a sting outed for it's incompetence by one mass media outlet (like a Fair Go for example) we could have trusted say a TV3 or a Stuff to carry it, but as this case involves the sacred cow Consumer, I'm not holding my breath.
On 10 June 2010 at 12:25 pm Tony Ryburn said:
"Seriously flawed" maybe. But the inconvenient truth is that most of the advisors gave poor advice.
On 10 June 2010 at 1:19 pm Independent Observer said:
A couple of related facts to ponder over:

Generally speaking, many NZ financial advisers do provide bad advice (relative to more mature jurisdictions where the advisory industry is further developed). Most NZ advisers are reliant upon substandard research or the tripe that has been produced by manufacturers to base their decisions

The NZ financial services industry (like all other industries) has its fair share of villains who are ruining it for everyone. Unfortunately the NZ Regulator has neither the competence nor the resources to put these folks out of business. So beware – many still work amongst us.

Despite the previous points, most of the NZ financial advisory community take their fiduciary responsibility to consumers seriously, and endeavour to provide the best level of financial guidance that they are aware of.

…all of which should be considered when deciding whether the IFA’s half price sale represents value or not.
On 10 June 2010 at 1:26 pm Kimble said:
Ummm, thats the thing Tony, because the study is flawed you dont know if you can believe its conclusions.
On 10 June 2010 at 1:30 pm David Whyte said:
What is truly inconvenient is that advisers MAY have given poor advice, but for the reasons cited in my posting - and in the references to Michael Mintrom's report - it is not possible to draw any meaningful conclusions.
On 11 June 2010 at 9:46 am alan said:
do we actually know who at Consumer deemed that plans were either good, bad or indifferent, and what qualified them to come to the conclusions they did?
On 11 June 2010 at 1:24 pm Mark Jory said:
Independent Observer - "genereally speaking, many NZ financial advisers do provide bad advice". How many NZ financial advisers have you sought personal financial advice from? There's about 1,400 members in the IFA, but perhaps several thousand others who are not members of that professional body.

To be able to come to that conclusion, either you have personally sampled a majority of the many thousands of financial advisers in NZ, or alternatively we can accept that statement with the same lack of competence as the Consumer report.

By the way, generally speaking, independent observers are never independent! Are they?

I also beleive you do not understand fully the current IFA "half-price" sale (as you call it).

Current members who do join up a new member to the IFA are entitled to a 50% discount on their own renewal memebrship. The member may choose to share some or all of that discount with the new member.

In other words, any new member joining the IFA does not automatically get 50 off their membership, that is up to the nominating existing IFA member to decide. At least that is my understanding of the offer.

Finally, all Financial Advisers ask of any report on the quality of our advice is a fair and reasonable report based on the same sort of professional standards of competence we are being asked to produce. Clearly this report was not up to an acceptable standard of competence.
On 11 June 2010 at 4:49 pm Bruce Woodley said:
Previous attempts to clean up the financial advisory industry have been unsucessful and now government is taking positive steps towards a competent industry.Millions of dollars have been lost by acting upon bad advice from these so called advisors. We should encouarge any attempts to improve an industry which is full of fraudsters in various shapes and disguises.
On 11 June 2010 at 5:52 pm Graeme Lindsay said:
Bruce Woodley. You comment displays a similar level of ignorance as a number of other previous comments.

I am not an investment advisor, but I know a great number of them, and in the main, they are competent people, who advise their clients with skill, integrity and a real concern for those clients. The industry is NOT full of fraudsters! Inevitably, there are some who do not meet the standards that are reasonably required of them, but, if you think about it, there are 'bad eggs' in every business or profession! There are no more in the investment advisory world than any other.

The lead article above addresses the fact that the Consumers Institute conducted an unscientific test that has been proven to be seriously flawed. It seriously calls into question their skills, integrity and motivation.

As an aside, I recall that on this website, a previous blog showed that far more people invested in finance company securities responding to advertisements of the promoters than did acting on the advice of a financial planner. They were responding to promises of high earnings and believed the claims of the promoters and their auditors. Sadly, no one in the finance company world had a crystal ball that enabled them to foresee the disastrous credit crunch that caused the collapse of many of the promoters. Clearly some promoters were crooks, but, I suggest, that most were working quite legally, and doing a good job!

It's easy to sit behind a computer screen making unjustified and unpleasant statements about financial planners from a position of ingnorance - with some more understanding of that world, you might be less inclined.

Cheers
On 12 June 2010 at 8:21 am nicole said:
The public and the Securities Commission need to understand that alot of the spotlight on bad advice should have been put on lousy products and their providers. Investment products have been through the process of leglisation, company directors (usually very experienced), auditors, trustee companies, ratings agencies long before they ever get to the financial advisor. Im not convinced that regulating financial advisors is going to make much difference if everyone else in the industry fails to do their job compentently. I think the focus should have been at the beginning of the chain rather than at the end and then the public would be much better off.
On 14 June 2010 at 8:41 pm w k said:
Re: Nicole - totally agreed. I've said this before, providers/suppliers of investment products should be tightly regulated and scutinised first. What's the point of having only highly qualified, compliant and honest advisors when the system allows products to be provided by unscrupulous operators who can write very attractive brochures that 'complied' with current regulations? Clients can invest in them directly without going through advisors, just in case you forget.
If all providers/suppliers of financial are well regulated, then it does not matter if advisors are inexperience or incompetent (not that I am advocating), clients will be safe regardless of which product they invest in.
Regulators have to ask themselves, are you putting the responsibilites of all investment products 100% on the shoulders of advisors? - Anything goes wrong, nail the advisors first, the Regulators are not at fault, and the unscrupulous directors get away scott free or a slap on the wrist with the millions or billions. That's the signal I am getting.
Btw, just wondered, how many on the committees have ever had experience practicing as an advisor? I am asking, because I don't know, and I also don't expect some CEO, with an MBA or PhD, sitting on many boards for many years in the manufacturing sector to advise the DHB how to run their medical facilities.
On 18 June 2010 at 5:30 pm Barry Erickson said:
Methinks advisors protestith too much. How many bad advisors are out there? Invite a name call online for transparency. Dare you?
On 23 June 2010 at 9:19 am Ricardo said:
My experience of some financial 'advisers' is similar to my daily experience of some 'journalists'. There are those that do their research (rare), and those who rely unconditionally on information supplied by those with vested interests. Until we as the public start getting interested in both fields, the reasons why things are the way they are, and actually question what is put before us, then we will continue to get mediocre results.
Commenting is closed

www.GoodReturns.co.nz

© Copyright 1997-2024 Tarawera Publishing Ltd. All Rights Reserved