Restrict Group and Channel Invites. You can choose who is able to directly add you to groups and channels in your Privacy Settings.

Users who are not allowed to add you will be able to send an invite link as a regular message.