When a customer walks in, we judge them. There is a human need to judge regardless of how much we discuss, teach, educate and instruct. The trait is no doubt adaptive as our ancestors survived because they were able to determine if friend or foe was approaching based on their looks, dress and carriage. The skills that were very useful on the eastern plains of Rift Valley are a hindrance in modern day restaurant settings.
It does matter, you can not know who that person is by what they look like, how they walk, talk or with whom they journey. Not everyone needs to be your customer however everyone is crucial to the viability of your restaurant. The deal that you have bought into when you created the restaurant was that you would give everyone exceptional services. Being aware that you prejudge helps to mitigate some of the notions you have and a great deal of the minutia that hampers your restaurant from providing exceptional customer service every time.