Hi Prahlad,
why are you using the stepper motor with encoder? I mean steppers can be precisely controlled by the processor. If your application is only 60RPM (or upto 200RPM) then why are you using encoder, better you use a variable as char or int(8 bit or 16 bit depends) and just add or subtract the number of steps in it.
If you want a Shaft position:---
Considering the Sanyo Denki motor is 1.8 deg step angle.
Always subtract this variable by 200(If you are programming the L297 in full step mode) or 400(if you are programming the L297 in half step) if it is greater than that value. You can use an additional bit for sign(say + for CW and - for CCW) then you can easily find out what your shaft position is based on the value in the variable by multiplying it with the step angle of stepper motor.

If you want number of rotations or you have converted rotary into linear position:---

then you can count the number of rotations in one more variable Considering the Sanyo Denki motor is 1.8 deg step angle.
After you give 200(Full step) or 400(half step) pulses to L297 then count as one revolution and use sign bit for froward / rverse direction.

If you are saying that your shaft vibrates then its because you are not getting perfect holding. How many phase stepper are you using?
If it is 6 leads then for 50kg cm motor the current required will be more than 2A/phase in bipolar connections. And then your L297 and L298 combination will not work effectively. You need to cascade multiple L298s or use seperate 2 MOSFET bridges.

Also if you are using the current control of L297 inbuilt chopper then use it on inhibit lines instead of A, B, C, D lines. This is one of the common problem while designing the L297 + L298 drive.

The most important thing is the basic phenomena of stepper is to exhibit hystresis (As Steave said, stepper moves in discrete step angles and so he suggested to put rubber type disk to your motors shaft) What I mean is it gives resonance around the step angle and then settle to the final position, which hampers the accuracy of stepper and this may be resulting in your encoders output error.

Regards
Sarang