There is no penalty for this. You are insisting on deleting it, so delete it, or better move somewhere else, online, so you can still make some $$$ from that work that was done. If you can change anchor text of the links, that's good, and I would make sure that no links get 404 errors, after you delete the content from the site.
If you want to delete it, delete it, and go from there. Like I said, it is not a $20,000 website, or anything like that.
__________________
|