NOT INTERESTED

Why is it hard to simply say "Not interested"? Is it really that difficult to tell someone youre just getting to know that you're not interested to continue whatever you are doing ?