This paper reports the measurement of [Formula: see text] meson production in proton-proton ([Formula: see text]) and proton-lead ([Formula: see text]) collisions at a center-of-mass energy per nucleon pair of [Formula: see text] by the CMS experiment at the LHC. The data samples used in the analysis correspond to integrated luminosities of 28[Formula: see text] and 35[Formula: see text] for [Formula: see text] and [Formula: see text] collisions, respectively. Prompt and nonprompt [Formula: see text] mesons, the latter produced in the decay of [Formula: see text] hadrons, are measured in their dimuon decay channels. Differential cross sections are measured in the transverse momentum range of [Formula: see text], and center-of-mass rapidity ranges of [Formula: see text] ([Formula: see text]) and [Formula: see text] ([Formula: see text]). The nuclear modification factor, [Formula: see text], is measured as a function of both [Formula: see text] and [Formula: see text]. Small modifications to the [Formula: see text] cross sections are observed in [Formula: see text] relative to [Formula: see text] collisions. The ratio of [Formula: see text] production cross sections in [Formula: see text]-going and Pb-going directions, [Formula: see text], studied as functions of [Formula: see text] and [Formula: see text], shows a significant decrease for increasing transverse energy deposited at large pseudorapidities. These results, which cover a wide kinematic range, provide new insight on the role of cold nuclear matter effects on prompt and nonprompt [Formula: see text] production.