Panic Disorder, which includes all the different forms of Anxiety, is by far the most common background cause of the often deeply unpleasant and even frightening symptoms of Depersonalisation. For those of us who experience severe and/or prolonged Anxiety at any point in our lives (70% by some reckonings) it is a very common symptom, so much so that doctors will often regard it as "normal".
As such, you should not regard it as yet another reason to feel anxious. Try to recognise it for what it is: a kind of defence mechanism where our minds and bodies try to protect us from harm, producing a kind of barrier between our conscious minds and the turmoil going on in the deeper parts of our minds, allowing us to "carry on" as normally as possible in the present circumstances.
We can soon be rid of these unpleasant feelings of Depersonalisation if we concentrate instead on the root causes of our Anxiety: tackle and reduce those and the Depersonalisation symptoms will disappear.
Note: Depersonalisation, as a symptom of Anxiety/Panic Disorder, can be exacerbated quite significantly by the use of stimulant drugs like caffeine, and also by other drugs like alcohol, cannabis, etc (see next Section).