That’s a complicated question. First, even if there is an objective world “out there,” it necessarily doesn’t exist anything like how we perceive of it with our arbitrary organic, macroscopic senses, nor how we think of it with our human, simplifying and modularizing abstractions. Anything we can know of, such as the world, is necessarily ideological insofar as we know of it, and presumably the world “out there” isn’t ideological. And, since any two things we can possibly compare and contrast with each other (in our minds, of course) must necessarily both be ideological, there’s therefore no greater schism than between the ideological, such as our knowledge of the world, and the ideological, such as the world-in-itself. So, even if the world exists “out there,” it may not exist in the way that we understand as being “physical,” therefore in that sense the physical world may only exist in our minds.
Also, it’s possible that things don’t exist when we’re not witnessing them. There’s no way to prove otherwise. Perhaps the whole universe is in a superposition of all possible states until we “collapse the wavefunction” with our observations. What exactly constitutes a “measurement” or observation that can collapse the probability wavefunction is ill-defined in physics, and there’s debatably an infinite regress problem where, whatever interference happens within a system to collapse the wavefunction, the entire system from the outside could still be an uncollapsed wavefunction, and if something from the outside collapses it by interfering with it, then the entire system including that interference from outside could still be uncollapsed from an even more-outside view, etc., so it makes sense that the buck could ultimately stop at consciousness observing it, perhaps one universal consciousness.
Another possible worldview is idealism (I wrote more about this here), which holds that everything is ultimately mind. In that case, what is the physical? Maybe it’s no more than a mental model we make based on the qualia we experience for whatever reason which is necessarily generated by mind, be it internally or externally. Or, maybe seeing the physical is actually just a mode of perception of life/spirit/mind, maybe some particular spirit/mind, afforded by our physical bodies. Of course, that raises the question of how we can have physical bodies if nothing is physical. The answer would be that we have bodies made of mind and/or spirit, or perhaps of the relationship between our minds/spirits and some other entity’s, like a soul that comprises the physical universe, and from the perspective of these vessels, our own vessels appear physical just as the rest of the world does.