1965 Mustang Starter question?

Whenever I put the keys into the ignition to turn the car over the mustang will do nothing. All I hear is one click from the voltage regulator. So far I have replaced the battery, red battery cable, and starter relay switch (solenoid). Still no luck. When I apply a screwdriver to the "hot" post on the solenoid to start it, the starter seems to stay engaged now when the engine begins to run. A terrible shrieking sound occurs. I'm not sure if mt next step is to replace the starter or could there be a problem with the ignition switch wiring??? Just throwing out some ideas. I'm a beginner and would appreciate the advice someone with more experience may have. I'm spent more money on this than I would have liked so far. Time to ask for help. Much appreciated. Again, when I insert the keys into the ignition all I still get is a click from the voltage regulator. When I byp the ignition and use the hot post on the relay switch to start the Mustang a terrible screeching noise occures. I think the starter is not disengaging. When the engine is turned off it sounds like the starter is "winding down".