I am a baby boomer, one of the many Americans approaching their dream retirement age. I am also one of the many baby boomers who has a little panic attack every time I look at my retirement accounts and my exposure to the whims of the stock market.

One of the questions that I often get as a financial planner (and one that I ask myself) is: Should I take money out of my retirement account to pay off the remaining balance on my home’s mortgage?

The answer: It may make you feel good, but it is an expensive decision.

Let’s set up a simple scenario: Say you have $100,000 in annual earnings. You are paying 4 percent on a $100,000 mortgage (about $2,900 net after deductions). If you also have at least $100,000 sitting in a retirement money market account earning significantly less than 1 percent annual return, paying off the mortgage sounds like a winner, right? After all, you are saving more than 3 percent a year in interest.

Until you look at the details:

• You need to withdraw about $143,000 from your retirement account to also cover the federal income tax on the $100,000 withdrawal. The extra income from the withdrawal will bump your marginal tax bracket from 28 to 33 percent.
• If you are under the age of 59½ you will have to pay a 10 percent early-withdrawal penalty.
• If you throw in Massachusetts state income tax, you will need closer to $150,000.
• In hindsight, that $150,000 needs to earn just a little less than 2 percent to match your annual after-tax cost of $2,900 interest for the mortgage. Increase in federal and state income tax: You will pay $42,000 in Federal tax and $7,500 in Massachusetts state tax versus $18,332 in Federal and $4,500 in Massachusetts tax.
• If itemized deductions including the mortgage interest exceed the standard deduction of $7,400 but they will drop below this if you pay off the mortgage, you will pay an additional tax of $1,120 (28 percent of $4000 mortgage interest).

If you itemize and your after-tax cost for that mortgage is 2.9 percent (which is typical), the question becomes: Can you earn a return greater than 2.9 percent on your retirement account? Probably (in the long term) if your account includes a mix of investments.

Before you make the decision, take a look at the facts:

• Use Turbo Tax’s Tax Caster to learn what your marginal tax bracket was last year. (If you want to know your tax and marginal bracket go to www.turbotax.com, click on Tax Calculators & Tips and get a no-cost lesson on your marginal tax bracket.)
• Use Turbo Tax’s Tax Caster to estimate what your tax will be if you withdraw enough after tax to pay off the mortgage.

Finally, look at your long-term investment strategy for your retirement years.

A Social Security benefit of $2,000 per month (your latest SS statement has an estimate) is equivalent to almost $300,000 invested in a risk-free guaranteed income retirement account (assuming you live to age 85.)

A defined benefit plan of $2,500 per month is equivalent to almost $400,000 in a risk-free guaranteed income retirement account.

This means that you have an annual income of $54,000; guaranteed, it will never run out.

Another $350,000 in other retirement accounts (say, a 401k) could generate an annual income of $19,000 for the next 20 years at 1 percent (but it is gone after the 20 years). If you invest the same amount for long-term growth and increase the return to 7 percent, your annual income jumps to $31,000; at 12 percent, it’s $42,000 — pretty close to your pre-retirement income of $100,000.

I know that I cannot afford to pay off my mortgage. At least after thinking about it I should be able to get a little more sleep at night. I also know that I cannot rely on fixed income accounts. It may be nice to get rid of the risk and worry but the cost to my lifestyle would be pretty significant. Before you jump at something that seems attractive on the surface, make sure to get all the facts. Talk to an expert, understand your risk profile, and have a Plan B.

David Milton is senior lecturer in finance at Bentley University