What about the broken plurals of Arabic? The verbal system of Ancient Greek or Sanskrit and Vedic? The handful of irregular forms in English isn't really worth mentioning (and if you know just a little bit about the workings of ablaut and umlaut and English historical phonology, there isn't that much irregularity left, anyway).

Objectively speaking, all languages are about equally hard: infants everywhere in the world take about the same time to learn their native tongue.