Malaysian police said 31 Filipino intruders were gunned down in Borneo by security forces on Thursday.
The casualties are the highest number in a single day since a Muslim clan from the Philippines took over a village last month, according to the Associated Press.
However, the Filipino group denied its members had been killed.
Malaysian Prime Minister Najib Razak rejected the ceasefire offer made by the Filipino militants staking a claim over the Malaysian state of Sabah, according to Reuters.
"We want the militants to unconditionally surrender and hand over their weapons," Najib told a news conference, speaking from a palm oil plantation in the conflict-stricken region. He warned that if the group did not surrender, the military would target them "for as long as it takes to eliminate them.
The clan is led by Jamalul Kiram III, who claims to be the sultan of the southern province of Sulu in the Philippines, which is predominantly Muslim.
Abraham Idjirani, a representative of Kiram's, said earlier on Thursday, "We are showing to the whole world that first the sultan wants to resolve this peacefully," according to the AP.
At least 60 people have been killed in Borneo during the month-long conflict, including eight Malaysian policemen.
"The secretary general is closely following the situation in Sabah, Malaysia," said a statement from the United Nations released on Wednesday, according to The New York Times. "He urges an end to the violence and encourages dialogue among all the parties for a peaceful resolution of the situation."
"We're not rebels," Kiram told reporters, according to Bloomberg. "We're not terrorists. I have not committed any crime." According to him, his group is loyal to the Philippines.
However, Philippines President Benigno Aquino vowed to press charges against Kiram.