Yes, use the paste. Very important to do so.
Spread a little on the top of the CPU and spread it evenly to cover the whole surface. You want a nice smooth covering, without any bubbles or gaps and about the same thickness as a piece of 80gm paper. j.