"If a faction in my party goes on committing atrocities against common people, many of whom are my loyalists; if beatings and arson go on unabated without any interference by party higher-ups, I will be forced to abstain from voting in the Rajya Sabha polls," Islampur MLA Abdul Karim Chowdhury said.