Problem with water and oil and RPMs.

It does sound like the head gaskets are bad. You can sometimes tell by smelling the exhaust if it is mostly coolant or oil getting in the cobustion chamber as it will smell like anti-freeze, kinda a sweet smell, or burnt oil.
In either case you can do a compression test and/or a coolant system pressure test to make sure that is the issue before you put the money into the repair.