The usage of chain rule in physics

Well, this is the most common thing for which mathematicians make fun of physicists. Because we don't bother to cancel out derivatives, and we "NEVER" check if we can imply some rule on our equations. The thing is, that almost all functions, which can appear in nature or real life systems are, in most times, continuous and differentiable. There are, for sure, some special cases. But for most simple tasks, eg. mechanic, this is quite valid.

So in the case of $v$. In order to define velocity, the object has to change its position in some amount of time. And furthermore, we don't have infinite speed in real life. This implys, that $dx/dt$ has allways that some non infinite value. From this it follows, that $v$ can be rewritten as function of either $t$ or $x$.

I am not sure if there is a special case or not, but for physicists it is not important, because in 99.9% this will be true. If there are special cases, they could be "obviously strange". You should have in mind, that at least in theory, we always check our calculations with experiment, so we have an experimental proof instead of a mathematical one (generally).

It is true, that in nature there is only one true independent variable, time. All others are "pseudo-independent". They are variables humans bless as independent in order to answer what-if scenarios and to establish mathematical models of systems byways of separation of variables. The common term for these "pseudo-independent" quantities is generalized coordinates.

Looking at a complex mechanical system, like a human launching a ball while riding on a skateboard. First, we decide what the degrees of freedom are and assign generalized coordinates to them. These are simple measurable quantities of distance, angle or something else geometrical forming a generalized coordinate vector $$\boldsymbol{q} = \pmatrix{x_1 \\ \theta_2 \\ \vdots \\ q_j \\ \vdots} \tag{1}$$ In this example there are $n$ degrees of freedom. All the positions of important points on our mechanisms can be found from these $n$ quantities. If there are $k$ kinematic hardpoints (such as joints, geometric centers, etc) then the $i=1 \ldots k$ cartesian position vector is some function of the generalized coordinates and time $$ \boldsymbol{r}_i = \boldsymbol{\mathrm{pos}}_i(t,\, \boldsymbol{q}) \tag{2}$$

Here comes the chain rule part. With the assumption that (2) is differentiable with respect to the generalized coordinates, and that contact conditions do not change due to separation, or loss of traction, the velocity vectors of each of the hardpoints is found by the chain rule

$$ \boldsymbol{v}_i = \boldsymbol{\mathrm{vel}}_i(t,\,\boldsymbol{q},\,\boldsymbol{\dot{q}}) = \frac{\partial \boldsymbol{r}_i}{\partial t} + \frac{\partial \boldsymbol{r}_i }{\partial x_1} \dot{x}_1 + \frac{\partial \boldsymbol{r}_i }{\partial \theta_2} \dot{\theta}_2 + \ldots + \frac{\partial \boldsymbol{r}_i }{\partial q_j} \dot{q}_j + \ldots \tag{3} $$ where $q_j$ is the j-th element of $\boldsymbol{q}$, and $\dot{q}_j$ its speed (being linear or angular).

The above is not a division of infinitesimals, but the multiplication of a partial derivative $\tfrac{\partial \boldsymbol{r}_i }{\partial q_j}$ with the particular coordinate degree of freedom speed $\dot{q}_j$.

Maybe you are more comfortable with this more rigorous notation using partial derivatives that what you have seen so far. The term partial derivative means, take the derivative by varying only one quantity and holding all others constant. This is what allows us to use pseudo-independent quantities $q_j$ for the evaluation of the true derivative with time (the one actual independent quantity).

The same logic is applied to higher derivatives as well

$$ \boldsymbol{a}_i = \boldsymbol{\rm acc}_i(t,\boldsymbol{q},\boldsymbol{\dot q}) = \frac{\partial \boldsymbol{v}_i}{\partial t} + \ldots + \frac{ \partial \boldsymbol{v}_i}{\partial q_j}\, \dot{q}_j + \ldots + \frac{ \partial \boldsymbol{v}_i}{\partial \dot{q}_j} \,\ddot{q}_j \tag{4} $$

The last part might be a bit confusing, but when you express it in terms of actual degrees of freedom it might be clear. Consider the degree of freedom $\theta_2$ and its time derivatives $\omega_2$ and $\alpha_2$. Then the terms $\frac{ \partial \boldsymbol{v}_i}{\partial \theta_2} \omega_2 $ and $\frac{ \partial \boldsymbol{v}_i}{\partial \omega_2} \alpha_2 $ are more clear I hope, as $\boldsymbol{v}_i$ depends on both the position $\theta_2$ and the speed $\omega_2$.

I like this question and there already some good answers. I'm not going to repeat those, but I wanted to add a couple of points focused on the second part of your question regarding "swapping" differentials.

The first is that the presence of a differential quantity is an abstraction that is usually only useful as an intermediate step in calculating something else. By that, I mean you never measure something like $\rho\ dV$ directly. You can only hope to measure:

1. The integral of that quantity $\int \rho\ dV$ over some volume (equivalently, you back off of the abstraction and measure $\rho \Delta V$ for some finite volume $\Delta V$) --OR--
2. The "ratio of differentials" (being deliberately loose for the moment), which in the limit is a derivative. So an expression like $f(t) dt = g(x) dx$ gets "divided through" to be $f(t) = g(x) (dx/dt) = g(x)v(t)$. We believe we know how to measure changes in quantities and gradients.

That's relevant to the second part of your question about "swapping" differentials because when that's done legitimately, it typically works because you're ultimately going to put that expression under an integral sign, and the notation conveniently reflects (some might prefer to say the notation is easily abused when applying) the integration-by-substitution rule $$ \int_a^b f(g(x)) g'(x) dx = \int_{g(a)}^{g(b)} f(u) du$$ which you could rewrite in Leibnitz notation for $u = g(x)$ and get the appearance that you are swapping or canceling differentials.

Since the integration-by-substitution rule is basically the chain rule in reverse, however, all of this begs your initial question of why the chain rule is valid in physics. For that, I refer back to the other already-good answers.

that means we can write velocity as a function of distance

That's not quite the intended meaning. Rather, the intended meaning is:

If you could write velocity as a function of distance in the domain of interest, then the equation would hold.

It's up to you to deduce whether that assumption can be met satisfactorily in the problem, but usually it's quite obvious that it can.

One way to see this is that you can artificially restrict the domain to the portion of space and time that is of interest and disregard the rest of the domain, and then argue that this assumption would hold there.
(Notice I basically just rephrased the continuity notion of a limit here.)

The only way for this to be false in your particular example is to have multiple velocities at a given point in time (or no velocity at all), which generally wouldn't make sense in the (continuous) everyday world we're familiar with.

And if the discussion is about some unusual boundary condition where you can't take a limit on all sides and show the problem is continuous, then you wouldn't read such a claim about that situation without some kind of other (implicit or explicit) indication as to why it's true.

In situations like these it might be good to take a step back and consider what we are actually looking at. In this case we are looking at some function $x$ as a function of time. So starting from this the only functions that are well-defined are \begin{align} x:\quad t\rightarrow &x(t)\\ v:\quad t\rightarrow &v(t)=x'(t) \end{align} We can rewrite our culprit $\frac{dv}{dx}$ in terms of these functions. \begin{align} \frac{dv}{dx}=\frac{dv}{dt}\frac{dt}{dx}=v'(t)\left[x'(t)\right]^{-1} \end{align} Here we used the chain rule and the fact that the derivative of an inverse is the reciprocal of the original function i.e. $dy/dx=(dx/dy)^{-1}$. Immediately we can see two illuminating things: firstly we can define the derivative $\frac{dv}{dx}$ because we can write $v$ as a function of $t$ and we can also write $t$ as a function of $x$. Secondly this derivative is only defined if $x'(t)\neq 0$ so there are some constraints in doing this.

Now let's take as an example $x(t)=bt^2$. We can calculate this in two ways. The first way is to first substitute $t(x)$ and then differentiate with respect to $x$: \begin{align} t&=\pm\sqrt{\frac x b}\\ \implies v(x)&=v(t(x))=\pm 2\sqrt{bx}\\ \implies \frac{dv}{dx}&=\pm\sqrt{\frac b x} \end{align} The second way is to use the chain rule. From the second equation \begin{align} \frac{dv}{dx}&=v'(t)\left[x'(t)\right]^{-1}\\ &=2b[2bt]^{-1}\\ &=\frac 1 t\\ &=\pm\sqrt{\frac b x} \end{align} Perhaps unsurprisingly these methods are equal. The second method makes it really explicit which functions are used but the first method can be obscured sometimes when $t$ is not mentioned like in your question.

The main takeaway of this answer is that these tricks have a formal proof behind them but often the author leaves this out for brevity. This way we can do more physics more quickly but these tricks shouldn't go at the expense of your fundamental understanding. When you feel this happens it might be useful to write down the functions you are using and on which parameters they depend and then you can try to proof these tricks. A nice summary of these tricks is 'differentials are not algebraic entities so you can't just switch them around in fractions but it turns out in most cases you can switch them around like that'.

What we are really saying is that there is some function $f$ which when composed with position gives velocity. We have:

$$ v(t) = f \circ x(t)$$

Taking the derivative:

$$ \frac{dv}{dt} = \left[ \frac{df}{dx} \circ x(t) \right] \frac{dx}{dt}= v \left[ \frac{df}{dt} \circ x(t)\right]$$