There are many ways of knowing; evidence is just the most base way, perhaps the crudest–or at least it’s crude to staunchly rely exclusively on evidence–although it’s of course the most effective and incontrovertible (except to the degree that evidence can be misinterpreted, of course). This is why so many people feel that they should base their beliefs solely on evidence and nothing more; it’s based in fear, it’s a retraction. There are various causes for this fear and retraction, depending on the individual.
Some are deathly afraid of the possibility of being wrong about something, on a social level. Believing in something without stark evidence could make them vulnerable, because they can’t support their belief in the face of someone who believes something else, probably strictly on the basis of evidence.
Some have seen the way others are often misled and wrong in their beliefs, such as those influenced by religion, popular misconceptions, sources of intentional deception, wishful thinking and other cognitive biases, etc. etc. They conclude that we just can’t know anything for sure except on the basis of evidence.
Some have recognized being wrong in their own thinking, for some of the reasons listed above, and hence fall back to (retract to) believing only what can be proven.
In actuality we believe many many things in daily life that aren’t and can’t be proven, but some people don’t really reflect on it much in their moment-to-moment activities and believe that they believe things only on the base of evidence. And they do, when it comes to the big truths, just not to the more trivial truths. And the characteristic that it takes on when it comes to the big truths is one of denial / rejection by default of anything mysterious, amazing, extraordinary, or potentially paradigm-changing, or that could get them ridiculed for believing in. It’s not simply a state of non-belief or open-mindedness about a given subject prior to the evidence; it’s still biased, just in favor of the status quo and the mundane.
And of course they’re not actually considering all the evidence available. Just like regular people are afraid of getting laughed at, scientists aren’t free to report positive results in parapsychology and the like because those who do get promptly ostracized from the community and thought of as nut-jobs, regardless of the solidity of their data. So those people who believe they’re basing their beliefs on evidence are actually basing their beliefs on the biases of the scientific community, in some areas of belief. The belief that the practice of science is infallible, vis a vis the fact that scientists are merely human in practice, serves only to reinforce some of the prejudices that already exist in society.
But back to the main point..
Some believe that evidence is the only way of knowing things because anything other way of knowing must be psychic, and psychism is axiomatically impossible because it defies the laws of physics. Of course it doesn’t—it just defies some precepts that the type of people who like science (and the scientific community at large) are typically enveloped in. You can’t actually get from the known laws of physics to a proof that something psychic/parapsychological/paranormal can’t happen or that there can’t be some kind of synchronistic, mystical or spiritual correspondance between two things, etc.
Granted, it is hard to know what’s really true that isn’t proven by evidence. People are notoriously bad at surmising such truths, which is why so many people believe so many crazy things and nobody really agrees with each other on anything they can’t touch. But it’s not fair to leap from the inability of most people to correctly surmise truth to a condemnation or just a dismissal of anybody, categorically, who says they know something that isn’t necessarily or wholly based on evidence. Some people are just more astute than others.
We are not isolated collections of neurons exchanging impulses around in our heads, with our only links to the outside world and truth being via our physical senses. On a spiritual and mental level, we are simply connected, like everything is somewhat “embedded” in everything else. And you don’t have to be a bona fide psychic to utilize this facet of the universe, everyone does through inspiration/imagination/”random” thoughts and intuition to some degree, and on a regular basis.
But I make it sound as if the only way of knowing other than via evidence is via psychism. This isn’t strictly the case. You can surmise things, perhaps intuively, based on experience and patterns, connecting the dots in the tapestry, in a way that you could say is ultimately based on evidence (being that it’s based on experience), but that isn’t directly based on evidence in the sense that the conclusion is provable from experience or in the sense that there’s a more-or-less one-to-one correspondence between a piece of evidence, or a formally identified collection of pieces of evidence, and a given fact interpreted from it. It’s modeling, it’s heuristics, it’s induction, it’s varying degrees of liberty in what one might call “jumping to conclusions,” all of which imply degrees of freedom in what’s concluded.
Of course, in reality an intuitive impression is neither all pattern matching nor all psychic, it’s an interplay of the two.
An article which may or may not be relevant to the question, or at least to some of my answer, is “Doubting Doubt,” which was written by a person with an IQ in the top 99.9999 percentile, and can be found in this PDF: http://megasociety.org/noesis/197.pdf