The concept of Theory of Mind (ToM) is central to many social robotic studies. It may be invoked during the design of social robots as a way to improve collaboration or create a form of “social intelligence”. It is also considered as an established fact and never put in question. However, many scholars have analysed the concept of ToM from a critical perspective and argued that it is, in fact, a theory with significant shortcomings, that rests on neuronormative and neuroprivileged grounds. In this paper, we explore these arguments and what this change of perspective means for the field of Social Robotics. We argue that the field should abandon the concept of ToM and move forward to more appropriate and inclusive models and research questions, and propose some directions to do so.