Let me be up front about my feelings for Warren Buffett: I really admire him. He is down-to-earth, lives modestly, likes to have lunch at Dairy Queen, and probably has no bad habits at all. He is also friendly, has a winning personality, he’s very grand-fatherly, and when he smiles it is almost a certainty that he means it. But I neither follow nor support his investment advice despite his net worth that is somewhere around $65 billion. I’ll tell you why, but keep in mind, I like him.

Warren gives this advice to all investors: Get out of debt and be a passive investor.

Get out of debt and stay out of debt is good advice. It has a few exceptions, however. Most of us need to borrow to buy a house (almost always a better choice than renting), also for a few other purchases to boost or to maintain our credit scores. Having excellent credit is more important today than is being completely debt-free. Most of us must also give up about 15 to 20 percent of our income to interest payments to others just to live at a moderate level of comfort. Still, I am on board with him for the most part on this issue.

His advice to invest passively, however, bumps up against everything I know about investing—and I do not yield one inch to Warren’s knowledge on this subject. Investing passively is only for those who don’t really need more wealth; it is not for those who would like to accumulate enough wealth to find their comfort level. There are many more people in the second group than there are in the first. On that point, Warren and I part ways. Here’s why:

The old maxim “nothing succeeds like success” is true about wealth. First, most of it is inherited, and then that inherited wealth grows to much higher levels, passively, and with the assistance of the tax laws. Therein lies the substance of Warren’s advice. But is it mainly for the wealthy, not for the average Joe who simply wants a small piece of the wealth pie. Indeed, the separation between those at the top of the wealth heap and those in the middle is so great, the poor sap in the middle hasn’t got a snowball’s chance in hell of ever improving his lot in life by becoming a passive investor. That is because his only real choice is to put his money into mutual funds and let it ride through good times and bad. And that virtually guarantees him or her mediocre returns, and virtually no chance for advancement to even the middle class. The compounding arithmetic is irrefutable:

Scenario #1. Assume an average income of $46,000. Start with $1,000 investment, add $300 per month for 30 years, receive a compound growth rate of 5% and you will have a grand total of $248,554. That accumulation will allow you to draw down $900 per month for the next 30 years provided you can earn a passive 2.3 percent per year during your retirement. And if you augment that income with Social Security, you will find yourself at the poverty level. So much for passive investing.

Scenario #2. Now leave everything constant except your rate of return. Boost it to 12 percent, and here is what would happen: Your total accumulation would rise to $944,555. and your monthly draw down would increase to $3,500 for 30 years, which would be approximately equal to your income during your working years, or more than double the poverty level.

The difference between earning 12 percent per year rather than 5 percent can be learned in one or two days of concentrated effort. And that is easily within the abilities of almost everyone.