Mr O'Neill also performed a reconciliation ceremony with the autonomous region's president, John Momis, and visited the site of the Panguna copper mine which sparked the civil war.
"Following custom, I'd like to say we are truly sorry for all the bad things that happened in your communities in Bougainville and our country Papua New Guinea," the Post Courier reported Mr O'Neill as saying on Tuesday.
Mr O'Neill made the comments at Bel Isi Park in Buka, where he and Mr Momis broke an arrow in a symbolic gesture of peace.
Mr Momis told a crowd of hundreds Mr O'Neill's visit meant a new beginning for PNG and Bougainville.
"This means a new beginning and cooperation and collaboration to continue the work for development," he said.
Mr O'Neill unveiled K1.5 million in development funds for Bougainville.
His visit marks the second by a PNG prime minister since Bill Skate in 1998, when both sides of the conflict brokered a peace deal.
Mr O'Neill brought with him the PNG government's chief secretary and Ministers Ben Micah and Byron Chan.
Mr Chan is the son of fformer prime minister Sir Julius Chan, who along with Mr Momis is considered one of PNG's founding fathers.
Bougainville is due to hold a referendum to decide if it will become an independent country between 2015 and 2020.
Australian Strategic Policy Institute analyst Karl Claxton said there is a wide expectation Bougainville will vote to become independent.
"(Mr O'Neill's) visit is definitely a welcome increase in focus and it's exactly what's needed, dialogue between the national government and the Autonomous Bougainville Government.
"I would call it a very significant step indeed."
Mr O'Neill on Wednesday is expected to visit the site of the Panguna copper mine near Bougainville's capital, Arawa.
At the time of its closure due to the civil war in 1989 the mine was the largest open cut copper mine in the world.
The reopening of the mine is still a hot issue in Bougainville.
However some argue it is a vital potential revenue stream for an independent Bougainville.
Mr Claxton said there is room for Mr Momis to stretch out the independence vote until 2020.
"To build consensus," he said.
"There is very little understanding of what autonomy means and how much is needed to make either of those things (autonomy or staying with PNG) work.
"Independence will need a big income stream."