Nature abhors a vacuum.
If we remove all the bad things in our life, that is good. But what if we don't replace them with something good? Eventually, Jesus says, all those bad things will return.
It's like removing all the junk food but not getting any healthy food... eventually, we'll go find more junk food, because we'll be hungry. We need to make it easy on ourselves to surround ourselves with healthy food.
In the same way, let us pursue Jesus, putting positive spiritual exercises and disciplines in our lives. When you have idle time, fill it with prayer or study. Read a chapter of Scripture. Fill your time and mind with thoughts and words of God.