The answer is yes.
Moreover the assumptions can be substantially weakened. Namely, it suffices to assume that each interval $(a,x_1),(x_1,x_2),\ldots,(x_n,b)$ contains a point, where the derivative is positive. In particular no limit assumptions at the points $x_k$ are needed.
Let $a<x_1<\ldots <x_n<b,$ where $f$ is not differentiable at $x_k,$ $1\le k\le n.$ Assume the conclusion does not hold, i.e. $f'(x)\neq 0$ for $x\neq a,x_1,\ldots, x_n,b.$ Then, basing on the Rolle theorem, the function $f$ is strictly monotone on every interval $[a,x_1],[x_1,x_2],\ldots, [x_n,b]$ (see the spoiler). Therefore the derivative is negative or positive on each interval $(a,x_1),(x_1,x_2),\ldots,(x_n,b).$ As each interval contains a point, where the derivative is positive, then the derivative is positive on each interval. Hence $f$ is strictly increasing on each interval $[a,x_1],[x_1,x_2],\ldots, [x_n,b].$ Thus $f$ is strictly increasing, which leads to $f(a)<f(b),$ a contradiction.
The following well known fact has been used in the answer. Assume $f$ is continuous on $[c,d]$ and $f'$ does not vanish on $(c,d).$ Then $f$ is strictly monotone. Indeed assume by contradiction, the opposite. Then the function $g(x,y)=f(y)-f(x)$ defined on $a\le x<y\le b$ admits nonnegative and nonpositive values. By IMV there are $x<y$ such that $g(x,y)=0,$ i.e. $f(x)=f(y). $ By the Rolle theorem $f'$ vanishes at some $z,$ $x<z<y,$ a contradiction.