CryptographyLevel 4★★★★☆140 pts

Cryptanalysis - Recovering an LCG's Hidden Parameters
MissionA service generated both session tokens and an encryption keystream from a linear congruential generator x_{k+1} = (a*x_k + c) mod m whose parameters a, c and m are secret. Recover a, c and m from the 10 consecutive outputs in tokens.txt, then decrypt secret.enc.hex, which was encrypted with a keystream made of the following outputs as 8-byte blocks, and extract the flag.
Hints
Hint 1Show
Taking differences t_k = x_{k+1} - x_k removes c, giving t_{k+1} ≡ a*t_k (mod m)
Hint 2Show
u_k = t_{k+2}*t_k - t_{k+1}^2 is a multiple of m. The gcd of several u_k values gives m
Hint 3Show
a = (x_3 - x_2)*(x_2 - x_1)^-1 mod m and c = x_2 - a*x_1 mod m. Generate the outputs after x_10 and lay them out as 8-byte big-endian blocks