The visual system consists of two major subsystems, image-forming circuits that drive conscious vision and non-image-forming circuits for behaviors such as circadian photoentrainment. While historically considered non-overlapping, recent evidence has uncovered crosstalk between these subsystems. Here, we investigated shared developmental mechanisms. We revealed an unprecedented role for light in the maturation of the circadian clock and discovered that intrinsically photosensitive retinal ganglion cells (ipRGCs) are critical for this refinement process. In addition, ipRGCs regulate retinal waves independent of light, and developmental ablation of a subset of ipRGCs disrupts eye-specific segregation of retinogeniculate projections. Specifically, a subset of ipRGCs, comprising ~200 cells and which project intraretinally and to circadian centers in the brain, are sufficient to mediate both of these developmental processes. Thus, this subset of ipRGCs constitute a shared node in the neural networks that mediate light-dependent maturation of the circadian clock and light-independent refinement of retinogeniculate projections.