Cement or concrete marks can be removed with careful use of diluted hydrochloric acid or a propriety cement remover, immediately followed by a thorough rinsing with clean water.
Please note that if a lime-based aggregate has been used, the acid will tend to dissolve this as well.