This idea works great sometimes. Other times you go bankrupt. 2008 is a good example of what can happen. House prices dropped, the stock market tanked and a bunch of people got laid off. In your situation you would still owe money on the house which wasn't worth what you paid for it. The equity you invested in the market is gone and you just lost your job. If you don't think that can happen then you either were not born yet in 2008, or you didn't pay any attention to what was happening around you.

Right now is an interesting situation to try out what you are suggesting. The market is down 20% but house prices are way up. If you think house prices are going to stay up forever then pull all of your equity out of your house and put it into the stock market. That way when the market goes up you'll make a ton. And if house prices stay high you won't have any problems. But, if the stock market goes down you'll lose part of your equity. If the stock market goes down and house prices drop and you lose your job then you might end up broke with no house.