I just installed the latest version of Skype on my Windows 10 Surface Pro 4 and what I saw was too many contacts added to my Contacts List.
I like to clean my Skype and machine very clean. I do not have too many contacts in my Skype list. And here is an interesting part, you cannot delete these contacts.
I guess I was logged into Skype using Online Outlook and that is where it imported my Outlook contacts into Skype Contacts list.
So what I end up doing is, disable Online contacts list.
Go to Skype Contacts > Contact Lists. You will see by default All is selected.
Just select Skype from drop down menu.
It doesn't sound like an optimal solution but it did the trick.
Com'on Skype. Don't be LinkedIn!