Bone marrow fibrosis (BMF) develops in various hematological and non-hematological conditions and is a central pathological feature of myelofibrosis. Effective cell-targeted therapeutics are needed, but the cellular origin of BMF remains elusive. Here, we show using genetic fate tracing in two murine models of BMF that Gli1+ mesenchymal stromal cells (MSCs) are recruited from the endosteal and perivascular niche to become fibrosis-driving myofibroblasts in the bone marrow. Genetic ablation of Gli1+ cells abolished BMF and rescued bone marrow failure. Pharmacological targeting of Gli proteins with GANT61 inhibited Gli1+ cell expansion and myofibroblast differentiation and attenuated fibrosis severity. The same pathway is also active in human BMF, and Gli1 expression in BMF significantly correlates with the severity of the disease. In addition, GANT61 treatment reduced the myofibroblastic phenotype of human MSCs isolated from patients with BMF, suggesting that targeting of Gli proteins could be a relevant therapeutic strategy.