Will only start with key turned and held in one position

My car will only start when the key is turned all the way and held in the same position for at least a few seconds. I tried replacing the ignition lock cylinder and I already replaced the starter.

Once I do get the car started, I usually do not even have the problem for the rest of the day and that is the weird part. I took the car to a mechanic to get the ignition lock cylinder replaced and he did mention something about the anti-theft system light coming on but I never noticed it being on.

I am at a total loss on what to do. I spent so much money into this car and it just keeps on giving me the same problems.