This paper considers a millimeter wave multiple-input-single-output (MISO) communication system assisted by one intelligent reflecting surface (IRS) placed at the edge of two small-cells. Our goal is to improve the communication system by allocating the IRS to assist one of the small-cells through a two-stage algorithm. In the first-stage, the algorithm selects which small-cell is assisted by the IRS. In the second stage, the phase shift of each IRS reflecting element and the beam-forming coefficients from each small-cell are computed. Unlike the majority of the solutions already proposed, the algorithm is designed to combine two distinct approaches. In the first stage, a data-driven reinforcement learning model is adopted to configure the general characteristics of each cell. While a traditional alternating optimization algorithm is adopted in the second stage to compute the coefficients associated with the IRS and small-cells. The performance of the two-stage algorithm is compared with three benchmark decision policies: random assignment, maximum capacity, and minimum capacity. Numerical results demonstrate the similarity between the communication system obtained through the proposed algorithm and the maximum capacity benchmark, confirming the effectiveness of the proposed solution to achieve a near-optimal capacity.