'Survivor' 's Dan Spilo Breaks Silence on His Behavior on the Show: 'I Am Deeply Sorry'

Dan Spilo is breaking his silence about his controversial time on Survivor: Island of the Idols — and apologizing that his actions made other contestants feel uncomfortable.

Spilo, a 48-year-old Hollywood agent, was removed from the game on day 36 after an incident in which he touched a female producer's thigh as he was getting into a transport boat — weeks after he was accused of inappropriately touching contestant Kellee Kim.

"I am deeply sorry for how my actions affected Kellee during the taping of this season of Survivor," Spilo writes in an exclusive statement to PEOPLE. "After apologizing at the tribal council when I first learned that Kellee still felt uncomfortable, I want to make sure I do so again, clearly and unambiguously."

"I truly regret that anyone was made to feel uncomfortable by my behavior," Spilo continues. "In my life, I have always tried to treat others with decency, integrity and kindness. I can only hope that my actions in the future can help me to make amends and show me to be the kind of father, husband, colleague and friend that I always aim to be."

It's the first time that Spilo has spoken out since his ouster from the show. Sources close to Spilo tell PEOPLE that he had planned to address the controversy during the show's live finale, but was later uninvited from the show.

Earlier in the season, Kim expressed concern that Spilo had violated her personal space by touching her hair, arms and legs, even after she made multiple requests for him to stop. Kim was voted out of the game after raising her concerns to other contestants.

Producers pulled each contestant aside to discuss the issue — but some players now say that the warnings were so unspecific that no one understood the severity of the issue.

"It was just a very vague blanket statement telling me if I ever felt unsafe, I should let production know," contestant Aaron Meredith told Parade. "There was never any indication that something was going on or that someone had issues."

As for the incident that led to his removal, according to multiple sources, Spilo insisted that the contact was inadvertent and accidental as he lost his balance while trying to get into the boat. But the show's production team wasn't convinced that the contact was merely incidental. After consulting with the show's legal team, producers removed him from the competition.

CBS is not providing additional comment at this time.

The season finale of Survivor airs Wednesday at 8 p.m. ET on CBS.