were wrong - children and old people have no more protection under the law than you or I when it comes to being photographed.
You may photograph anyone in a public place without his/her express consent, and may publish the photograph with full copyright protection. You own the copyright in the image, whether or not the subject gave his/her consent.
So far so good, but.....
The owner (or leaseholder) of a private property has the right to forbid photography, even if the building is freely accessible to the public. Local authorities (and Parish councils) may forbid photography in their buildings or on their land.
Unless you are expressly forbidden to publish the photographs by the owner of the property you may do so with impunity, at least as far as the law is concerned you may. In small communities it may be wise to take a more tactful approach, especially if you have to meet the subjects of your photography in the Post Office, or the local pub.
Technically the two people who objected were wrong however - there's no special protection for children or old people, and you do not need anyone's express permission.