As silly as it sounds, if the sun was on that one wall while you were grouting the others, that small amount of heat difference could have caught you out if it's so sensitive to heat.
Whatever you try on this, test in a super small area and wait for it to dry and whatnot before going ahead with the rest. Might even be worth testing it, then waiting for it to dry, and wetting the area to see if water makes any difference etc because sometimes it can look fine but then when steam hits it, it all comes back to a degree.
So whatever you do, test it first a fair bit.
Thanks for replying to this lads. He PM'd me worrying that he wasn't getting a reply. Then all of a sudden you'd clocked it.
Panic over hopefully.