Hi folks, grouted up 25m2 of slate floor today, and had a nightmare

. I have already done a load in the same house with no problems. Problem today being warm tiles/ warm weather.
The floor was cleaned plenty prior to grouting (I even use residue remover in my tiling water), and sealed twice with Fila Slate sealer. I have cleaned and scrubbed and cleaned and scrubbed. Then came to the conclusion I need something stronger and will go back in a couple of days when the grout is 100% gone off.
Has anyone else encountered this? I'm thinking Lithofin Cement away, either by hand and scrubbing brush action, or hiring a floor scrubber/ polisher to do the job - what would you do?
And then I'll seal 2 or 3 times with Fila again so it matches floor in room next door!
Cheers
Bob