With most laptops, when you plug in the charger I pilot light comes on to show it is connected and powering the battery. DO you see this light? If not I would suspect the charger has failed. Also, the icon on the bottom right of your screen should change to show that you are now running on mains power rather than battery. If you get both the pilot light and that changed icon I would suspect a faulty battery.
Surface Pro (2017) vs Surface Pro 4