Yes is the short answer.
But I have always thought that a USB drive was only a temporary storage medium so there would very rarely if ever be the need to defrag them.
Generally speaking,there is no need typically to be defragging one a week or even once a month. And with more modern OSs and disk formats, there is even not much need to do it once a year, especially if your drive it not very full. But, it can be worthwhile if you have a drive that is rather full, especially if you are using an older OS/drive format