You can see quite clearly in the first image that due to the lock-on targeting, not only is the ray spell nowhere near the cursor, it's blatantly missing. The second image shows that even though the ray is clearly "hitting" the pile, it is in fact missing and not triggering the pile's destruction. Sweeping around these piles also failed to hit them.
Earlier in the same game, I also had a fallen I had targeted able to walk through the beam, get into melee range, and become unhittable even by sweeping around.
Other abilities, such as Arcane Orb, do work as intended, and have been my go-to workaround for this issue.
EDIT: I'd also like to add that this can't be intended functionality, as I've had Ray of Frost stop on random NPCs (e.g. Necromancers during the Act II events) and other breakable objects.