I pay for my family’s cost of living in Thailand using two sources: 1) deferred compensation from my prior job and 2) cash withdrawals from my investment portfolio. My 2023 income from deferred compensation actually paid for all of the living expenses in Thailand, while the investment portfolio grew substantially. In other words, I am actually growing my wealth in mini-retirement! In this post, I intend to show how I was able to do this with my two sources of income.
First Income Source: Deferred Compensation
Almost a decade ago, I sat down with the Director of Compensation at my prior company. She described a benefit that I had as an executive: a SERP. She went on to explain that it’s a way to defer my compensation and reduce my tax exposure. At the time, I did not enroll in the benefit, thinking, ‘why in the world would I want to defer my pay? Didn’t Finance 101 say that money now is more valuable than money later?” But after seeing the effect that taxes had on my take-home pay, I reconsidered. Now mini-retired in Chiang Mai, I am very thankful for enrolling in this program and saving as much as I did before I resigned from my prior company.
- Benefit #1 from deferring compensation: Growing my portfolio tax-deferred
SERP is similar to a 401(k) in that the portfolio value is using pre-tax dollars, thereby giving you a larger portfolio value before you’re taxed. To give you a sense for how sizeable this is, the value of my SERP portfolio was roughly $280,000 at the start of 2023. I generated about $19,000 in returns from 100% dividends from the portfolio, or 6.8%. If the $280,000 were instead invested using after-tax dollars, the portfolio value would have been $180,000 (because of 37% marginal taxes I would have paid for that salary gained). Assuming same 6.8% return for those after-tax dollars, that’s $7,000 of value I gained in 2023 by using SERP! - Benefit #2 from deferring compensation: Reduced taxes
My tax rate in mini-retirement is substantially lower than my prior rate as an executive. While I was at the max bracket at 37% with my marginal income, my tax rate now is at 12%. Because I withdrew from my SERP last year at this lower tax bracket, I saved over $18,000 in taxes alone! - Benefit #3 from deferring compensation: Very low-risk income stream in mini-retirement
The risk of receiving this income in mini-retirement largely stems from the underlying risks of the SERP portfolio investments plus the financial viability of the company (where I was previously employed). Knowing I will be withdrawing from SERP in the early years of my mini-retirement, I moved all of my SERP to a low-risk investment: FXNAX, a low-fee index fund comprised of US Treasury Bonds. This fun basically throws off cash dividends, which are used to re-invest in my SERP. As for the risk of my prior company, I am very fortunate to have worked for a very financially sound publicly traded company with very healthy balance sheet and strong stock performance. Thankfully, the risk of my prior company going into bankruptcy is ultra low.
In summary with SERP as an income source, it paid for almost all of our costs of living in Thailand. In 2023, I netted about $66,000 from my deferred compensation (after paying for taxes). This pretty much covered the $72,000 spent on living in Thailand.
Second Income Source: Investment Portfolio Returns
In 2021, I decided to mini retire because I reached the point where I was financially independent. I believed that in addition to saving enough (emergency cash + deferred compensation), I had grown my portfolio to generate enough cash to more than offset my family’s estimated cost of living. You may have heard of the 4% rule that is used in retirement planning, whereby you estimate how much portfolio investment dollars you need my dividing your annual cash expenses by 4%. I used very conservative numbers. I thought we would spend $100,000 cash (even living in Thailand!); with a 4% hurdle rate (or minimum required rate I would expect from my portfolio), I needed $2.5 million in portfolio dollars. I am very grateful that I achieved that goal in 2021 and pulled the trigger to mini retire.
I’m happy to report that the 2023 realized returns off of this $2.5 million portfolio was roughly 18% – more than 4x the required return I needed to pay for our cost of living. Because my SERP paid for almost all of our actual living expenses, I simply re-invested these funds into the portfolio, making it more probable that the portfolio would generate the required funds for future years.
There are several important benefits to sourcing income from an investment portfolio:
- Benefit #1 from investment portfolio returns: Ultra passive income
I considered a variety of income streams to fund mini-retirement, everything from real estate to starting my own online business. Investment portfolio returns are basically the most passive of income streams, because it requires almost zero time and effort. I take roughly a few hours a quarter to do two things: 1) rebalance my portfolio and 2) make sure that I am selling investments at a loss to offset any investments sold at a gain, in order to reduce my tax exposure. I won’t get into why or how I do these two activities on this blog entry; the takeaway here is that the time freed up by placing almost all of our family’s funds into an investment portfolio is very significant. It was the biggest reason why to take the mini retirement in the first place! Rather than working behind a computer, I can spend my time traveling, learning new skills, being with my family, etc. - Benefit #2 from investment portfolio returns: Flexible risk profile
Income is never 100% guaranteed. It all depends on the risk you’re willing to take. While I was willing to take very low risk for the deferred compensation, I was willing to take more risk with my investment portfolio. I won’t get into the specifics of how I was able to generate an 18% return with my investment portfolio last year on this blog entry; I just want to make the point that the investment portfolio income stream is highly flexible with one’s risk appetite. I chose a diversified mix of index funds that fit my risk profile, and was very fortunate that 2023 rewarded some of the areas of the portfolio. But I could have easily modified my portfolio to be lower risk if I so desired. You can’t really do this with, for example, real estate investments, where the risks are quite inherent in the property you purchase. For the single family house we still own in Texas, we lost a lot of value from that property in 2023 because we didn’t foresee the risks assumed (e.g., tenant moving out without paying last month’s rent, multiple repairs made, deductible on hail damage incurred). In my opinion, the risks in an investment portfolio could be far less than those in placing funds into a few single family homes. - Benefit #3 from investment portfolio returns: Minimum taxes paid
Taxes are very low, if not zero, if one keeps tabs on the capital gains and losses incurred. Unlike ordinary income, income from investment portfolio returns are based on short- and long-term capital gains (or losses). So long as I offset any gains (from “winner” funds sold in my portfolio) with commensurate losses (from “loser” funds), then I effectively can withdraw funds from my portfolio without paying taxes. Keep in mind, for the majority of my investment portfolio, I already paid taxes on it when I got paid from my prior company. So that’s why I only pay taxes on the portfolio gains. Paying minimal taxes with your portfolio can be very significant because the portfolio grows unburdened with tax losses. To illustrate this with a quick example, say I have a portfolio of $1 million comprised of two funds. Half of the portfolio is in a “winner” fund that gains 12% in one year, while the other loses 6%. If I need to withdraw $30,000 from the portfolio (to pay for my cost of living), I sell $10,000 of the winner and $20,000 of the loser. The capital loss from selling the loser = 6% x $20,000 = $1,200. The capital gain from the selling the winner = 12% x $10,000 = $1,200. You guessed it – the loss fully offsets the gain. Your ending value of the portfolio = $560,000 + $470,000 – $30,000 withdrawn = $1,000,000. So you have effectively withdrawn $30,000 from your portfolio, tax-free and still have $1,000,000 in your portfolio. How’s that for tax-free income!
Summary
I want to be clear: I am not recommending these two funding sources or a specific investment strategy. I just wanted to show you how I am able to fund for my mini-retirement and actually grow my net worth at the same time. I was very blessed last year with a portfolio allocation mix that generated 18% returns, which could have lost money if external factors were different. But with careful planning, I believe that deferring my compensation largely funded my family’s costs of living in Thailand, while the portfolio returns was the cherry on top that helps secure more income for our future. As with any important financial decision, please consult a financial advisor if you are interested in doing something similar with your hard-earned money. I want to highlight that it can be done – and perhaps give you a little confidence that if your dreams are to take a break from work and grow your net worth at the same time, it’s achievable with proper planning and execution.
~Lester T