1992 Nissan 240SX Knocking and grinding noise

When I turn on my car while holding the clutch I can feel it grind a little. When it idles I hear like a knocking noise from my tranny. The noise is from the middle of my car not my engine for sure. When it idles it knocks, but when I push the clutch down it goes away. I think it's not my throwout bearing because you should already hear it grinding a little alredy. Plus, it should make noises when I push the clutch down. Right? Well, when I drive and shift gears it feels like a grind everytime I let go of the clutch and push it. I dont hear the knocking noise no more though when I drive. You can just feel the clutch pedal grinding everytime you shift. It's weird. I think the knocking could be from my clutch fork knocking agianst the housing. But when I drive it grinds. Help me. Thanks.