I do believe our early life experience affects what we grow to be, and this has been shown in studies. However, it's considered the first few years of our lives have the most significant impact on us, which is a few years before your experience with the elder man. This means your core personality had already formed before his involvement, and he would have had a minor affect.