You need to empty the 'Unsorted Bookmarks' folder, you can't delete it. It is supposed to be there.
In the top bar (File Edit View History Bookmarks etc...) click on 'Bookmarks' and then 'Organise Bookmarks' Then in the left pane click on 'Unsorted Bookmarks'. They will now be displayed in the right pane.
Now left click anywhere in the right pane, next hold down the Ctrl key on your keyboard and press 'a'. You should now have all the items selected. Rightclick on mouse and then choose 'Delete'