That’s a complicated question. First, even if there is an objective world “out there,” it necessarily doesn’t exist anything like how we perceive of it with our arbitrary organic, macroscopic senses, nor how we think of it with our human, simplifying and modularizing abstractions. Anything we can know of, such as the world, is necessarily ideological insofar as we know of it, and presumably the world “out there” isn’t ideological. And, since any two things we can possibly compare and contrast with each other (in our minds, of course) must necessarily both be ideological, there’s therefore no greater schism than between the ideological, such as our knowledge of the world, and the ideological, such as the world-in-itself. So, even if the world exists “out there,” it may not exist in the way that we understand as being “physical,” therefore in that sense the physical world may only exist in our minds.
Also, it’s possible that things don’t exist when we’re not witnessing them. There’s no way to prove otherwise. Perhaps the whole universe is in a superposition of all possible states until we “collapse the wavefunction” with our observations. What exactly constitutes a “measurement” or observation that can collapse the probability wavefunction is ill-defined in physics, and there’s debatably an infinite regress problem where, whatever interference happens within a system to collapse the wavefunction, the entire system from the outside could still be an uncollapsed wavefunction, and if something from the outside collapses it by interfering with it, then the entire system including that interference from outside could still be uncollapsed from an even more-outside view, etc., so it makes sense that the buck could ultimately stop on a macroscopic scale at conscious observation: if a consciousness were in a superposition, each possible state of consciousness would see itself and its surroundings as being in only one state.
Maybe there’s even a hierarchy of higher and higher consciousnesses on different levels of unity of beings affecting deeper and deeper levels of reality all the way up to the one universal consciousness, which is why we all seem to exist in a reality all of us can agree upon. But I have an alternative explanation for why we all seem to exist in the same reality somewhere in this essay.
Or maybe I’m just looking at it wrong and consciousnesses’ collapsing of wavefunctions would naturally be relative to each other because from each mind’s perspective only one possibility of each other mind’s perspective is seen. Of course, that doesn’t explain the consistency and persistence of underlying reality, but maybe that’s due to the non-stochastic aspect of quantum mechanics. I don’t know, this is all obviously over my head because I don’t know the actual physics behind it. I’m just rambling.
Another possible worldview is idealism (I wrote more about this here), which holds that everything is ultimately mind. In that case, what is the physical? Maybe it’s no more than a mental model we make based on the qualia we experience for whatever reason which is necessarily generated by mind, be it internally or externally. Or, maybe seeing the physical is actually just a mode of perception of life/spirit/mind, maybe some particular spirit/mind, afforded by our physical bodies. Of course, that raises the question of how we can have physical bodies if nothing is physical. The answer would be that we have bodies made of mind and/or spirit, or perhaps of the relationship between our minds/spirits and some other entity’s, like a soul that comprises the physical universe, and from the perspective of these vessels, our own vessels appear physical just as the rest of the world does.