I use a breakout board with my SS on my machines, more for the convenience of connection but also as a bit of added safety. I use the PMDX 122.
One thing I will recommend is that you convert the single ended step/dir outputs of Mach to differential, it makes them much more immune to noise issues.
Increasing the gearing mechanically will give you better resolution, think of it like this, if your encoder was 10,000 steps per rev and was direct coupled you would have 10,000 steps per screw revolution but if your gearing was 4:1 then it would be 40,000 per screw rev so a 4 fold increase in resolution.
Ok your encoders are sine/cosine and the default is 1024 counts per rev, you can multiply this in the drive to get over 1 million counts per rev if you wish but no need for anything like that
I have these kind of encoders on my lathe and mills spindles and I just have them set for 1024 but if it was an axis I would probably be looking at multiplying by 8. If I recall you can multiply by 2, 4 , 8, 16, 32, 64, ...... up to 1024
Hood