There are no ferries from Marmaris to Kalkan.
The most economical way is the midi buses (Bati Antalya, et al.) Cost would be around 35TL pp for both legs this year. (This is a guess-timate based on last years prices to Fethiye from Kalkan and considering the distance from Marmaris to Fethiye being about the same.)
Regret, my knowledge of the big overnight bus routes is not sufficient to say positively but, when the big bus gets to Kalkan at 10.00, IF it does call at Marmaris, it would be at around 06.00.