2001 Volvo V70 Turn Ignition Key and Nothing Happens

On more than one occassion I have had the following happen:
I drive the car somewhere, turn it off, go into a store or something, come back out, insert the key, turn it and nothing happens. No engine starting, no click as if a dead battery, etc. All the lights are on but there is no action whatsoever from the engine. So I have tried to jump it thinking it was the battery, but still nothing when you turn the key.
If I let the car sit, which I have had to do, and come back hours later and insert the key and turn it, it starts right up! I have actually had it towed to Volvo to have them diagnose the problem in that condition (the nonstarting condition) only for them to take it off the tow truck, insert the key and turn it, it starts up! Volvo has no clue what the problem is. Do you have any insight!