There is thin ice near the edge, but the water isn't deep. With boots you could probably wade the fifty feet and explore the back reaches. We plan to return in summer with our kayaks, paddle among the pond's cattails and see what it's like inside the cave.
To skip ahead, we returned the following day by car with our oldest boy, who was obviously lagging after a late night. The temperature had risen to 50F, not ideal for moving across the pond's surface, which had turned to slush overnight. I quickly moved to the opening.
He was fascinated with the rock. He made this mom nervous, of course. But boys will be boys and, fortunately, he is the cautious one.