Yes, you should if it is over the maximum limit. I stand to be corrected but from memory the limit is ~600mls cylinder capacity = 'lecture' sized bottles 12" x 2". But be aware that you wont be allowed down some tunnels if you have any hazard symbols displayed.
on a private vehicle if you carry less than 1000 litres of compressed oxygen, although you should carry a TREM (TRansport EMergency) card. This details the nature of the material carried, and actions to be taken by the driver and/or emergency services in the event of an accident. You can get an oxygen TREM card from BOC,ask for an 842 card.
Although you're not required to display a green diamond you might consider the fact that in the event of an accident in which you were left unconscious the emergency services would have no way of knowing what was aboard - they would proceed as if yours was a normal car, and if your vehicle caught fire the consequences could be much worse.
I think that once you use the ADR signs, you must comply with the full regs click here so, bear in mind that if you're not carrying oxygen at any time a TREM card must be in an envelope or similar storage cover clearly marked 'TREM cards not in use'.
This is so that in the event of an accident, the firemen know what substances are 'not' aboard.