Personally, unless it there is an ID on the part, I would work from outside to centre when facing.
That however is not your question, what you will need to do is have the Z clearance just marginally larger than your roughing cut depth.
For example if you have a rouging depth of 0.5mm then set the clearance to 0.501 and you should get more or less what you want.
Hood