$\because$点$P$关于$x$轴的对称点的坐标为$\left(2a+b,-a+1\right)$，
$\therefore$点$P$的坐标为$\left(2a+b,a-1\right)$，
$\because$关于$y$轴对称点的坐标为$\left(4-b,b+2\right)$，
$\therefore$点$P$的坐标为$\left(b-4,b+2\right)$，
则$\left\{\begin{array}{l}2a+b=b-4\\ a-1=b+2\end{array}\right.$
解得$\left\{\begin{array}{l}a=-2\\ b=-5\end{array}\right.$
$\therefore a-b=3$.

$\because$点$P$关于$x$轴的对称点的坐标为$\left(2a+b,-a+1\right)$，
$\therefore$点$P$的坐标为$\left(2a+b,a-1\right)$，
$\because$关于$y$轴对称点的坐标为$\left(4-b,b+2\right)$，
$\therefore$点$P$的坐标为$\left(b-4,b+2\right)$，
则$\left\{\begin{array}{l}2a+b=b-4\\ a-1=b+2\end{array}\right.$
解得$\left\{\begin{array}{l}a=-2\\ b=-5\end{array}\right.$
$\therefore a-b=3$.