Former rebel fighters battled their way into the heart of this coastal city on Sunday, seizing a sumptuous conference center used by Col. Muammar el-Qaddafi to entertain heads of state, but more recently by his loyalists as a base for their fierce defense of this city, his birthplace.