I ran into error this a little while ago while doing some data conversions.
A quick search on Google will yield results on trying to associate an invalid activity record to your entity.
However this was not the case, as I was not doing any linking of activity records.
In the end what it turned out to be was that the relationship was created against Entity A (which was wrong), while I was passing in an EntityReference to Entity B.
Still wrong, still an error, but took me down the wrong path when the error first popped up.
And now you know.