This letter proposes a new algorithm of refining the quantization parameter in H.264 real-time encoding. In the H.264 encoding, the quantization parameter computed according to the quadratic rate model is not accurate in meeting the target bit rate. In order to make the actual encoded bit rate closer to the target bit rate, ρ-domain rate model is introduced in our proposed quantization parameter refinement algorithm. Simulation results show that the proposed algorithm achieves obvious gain in PSNR and has stabler encoded bit rate compared to Jiang's algorithm.