I'd call it off, let him call me a coward or whatever, waste his time ripping into me again, and just get on with things. It sounds like you've got more to worry about in LA than defending your reputation.

Had you considered relocating to Montreal at all? It's quite cheap living here (some sublets or apartments can be had at $450 Canadian per month with utilities, a studio downtown with all utilities included is ~ $650) and you'll find more like minded company, and still be able to partake in some heated debate if you like.