I hate to say "only" about things like this because there are nearly always exceptions, such as size differentials or disease vectors or even just normal misunderstandings that can sometimes lead to difficulties and need to be taken into account.

But pretty much, yes. By far the chief danger is from societal attitudes.