Programmatically delete form controls by name or part name

Question

I try to write by using VB.Net 2010 an app that allow programmatically (from code) creating and deleting some of controls, TextBoxes. After few miss I made a part that create wanted (1-9) quantity of TextBox controls under name MyTextBox1.... MyTextBox2 but when I wish to delete just these TextBoxes by the its name I can't do that! Why I need to delete lready created Textbox? Simply, if I create for first time 5 controls on form and I change my mind and wish rather just 3 I must at first to delete existing controls before creating a new series.  On the form are other important textboxes (Name, Lastname) I wish to keep there but just delete these which name start with MyTextBox*!
In some moment I deleted just even forms's controls of TextBox and after I repeat click on button for delete code deleted odd number of remained textboxes!Any help is welcome.