The Jawbone was taken from the Speaking Demon in the Way Station. Roland Deschain took it, knowing that it could prove useful when dealing with similar demons in the future.
Roland gave the jawbone to Jake when they approached the Oracle in the Mountains. He knew that the Oracle would call to Jake and that Jake would have a hard time resisting that call.
The jawbone was later lost, but Roland gained a second one. He believed it was Walter O'Dim's, but Walter appears later, much alive.
Roland throws the second jawbone into his ka-tet's fire, at which time Eddie Dean sees his key.