Simply the answer is Yes. The full print resolution for a full a4 page is about 120 megapixels on an average photo printer. Clearly you are not going to (and dont need to) take photos at that resolution because the print software fills in. What you do need is a matter of judgement but I would say the minimum for a 6x4 print is 3 megapixels and you might just get a passable a4 print at this level. Be aware that the quality of the paper is just as important as the quality of the printer or picture when printing photos. I could not beleive how true this was until I did some tests. On ordinary paper it not only looks grainy but much of the detail disappears altogether.
Hope this helps