Residents said they had no choice but to flee Sporong because illegal miners regularly invaded their homes, assaulted them and demanded cash.