Transmission will drop out of gear

After driving a few miles the transmission will drop out of gear. If you let it sit for a few minutes it will go into gear and do fine for a quarter mile or so, depending on how long it sits. I am changing the fluid and filter and checking the valve body. Heat obviously causes the problem. Before I put it back together I would like to know if I have to rebuild it. When it drives normally everything is fine. The converter shutters a little once in a blue moon and I just have to let up on the gas a tad and it stops and is not a problem. The only thing I found in the pan not normal was some of the sludge was hard as a brick. It acts like the pressure valve is just dropping pressure. I can rebuild it but wood like to know what I should look for. Thanks for your help! Hank