You can go through China. I did the reverse-- take a bus from KZ to Urumqi, and then catch a bus (no direct bus, you'll need a connection) to the border at Takeshikenzhen. You can walk between the border points easily, and catch a car or truck to Khovd, or wherever you want in Mongolia. Sorry, I don't remember the prices, but it should be pretty cheap. Good luck!
thank you for your nice reply and its good to visit west of china but are you sure about the border and their rules? and also its very far to Ulaanbataar, and how cn i reach there? can you give me more information about it? i am very certain to do this plan. thank you.
i've never been to the west Mongolia, but it is really far from UB. and that is the Khazak people living in the mountainous Altai area. bus will be the only way out to UB. no trains. It will take about 2 days and a night. and the bus is not a sleeping seat, quite tough on the bumpy way. I suggest you to stay in Altai area for a couple of days. it is beautiful. just make sure you have warm clothes.
hey, couple of our couch surfers travelled to Ulan-Bator, and it was through Russia and China. as you can see on the map, Kazakhstan doesnt have a border with Mongolia. So probably you need to go to Russia and take a train. if you need info about Kazakhstan you can check http://www.caravanistan.com/transport/train/kazakhstan/