Almost everyone... seems to be quite sure that the differences between the methodologies of history and of the natural sciences are vast. For, we are assured, it is well known that in the natural sciences we start from observation and proceed by induction to theory. And is it not obvious that in history we proceed very differently? Yes, I agree that we proceed very differently. But we do so in the natural sciences as well.
In both we start from myths—from traditional prejudices, beset with error—and from these we proceed by criticism: by the critical elimination of errors. In both the role of evidence is, in the main, to correct our mistakes, our prejudices, our tentative theories—that is, to play a part in the critical discussion, in the elimination of error. By correcting our mistakes, we raise new problems. And in order to solve these problems, we invent conjectures, that is, tentative theories, which we submit to critical discussion, directed towards the elimination of error.

Another argument of hope may be drawn from this–that some of the inventions already known are such as before they were discovered it could hardly have entered any man's head to think of; they would have been simply set aside as impossible. For in conjecturing what may be men set before them the example of what has been, and divine of the new with an imagination preoccupied and colored by the old; which way of forming opinions is very fallacious, for streams that are drawn from the springheads of nature do not always run in the old channels.

But I should be very sorry if an interpretation founded on a most conjectural scientific hypothesis were to get fastened to the text in Genesis... The rate of change of scientific hypothesis is naturally much more rapid than that of Biblical interpretations, so that if an interpretation is founded on such an hypothesis, it may help to keep the hypothesis above ground long after it ought to be buried and forgotten.

Firm support has been found for the assertion that electricity occurs at thousands of points where we at most conjectured that it was present. Innumerable electrical particles oscillate in every flame and light source. We can in fact assume that every heat source is filled with electrons which will continue to oscillate ceaselessly and indefinitely. All these electrons leave their impression on the emitted rays. We can hope that experimental study of the radiation phenomena, which are exposed to various influences, but in particular to the effect of magnetism, will provide us with useful data concerning a new field, that of atomistic astronomy, as Lodge called it, populated with atoms and electrons instead of planets and worlds.

In all speculations on the origin, or agents that have produced the changes on this globe, it is probable that we ought to keep within the boundaries of the probable effects resulting from the regular operations of the great laws of nature which our experience and observation have brought within the sphere of our knowledge. When we overleap those limits, and suppose a total change in nature's laws, we embark on the sea of uncertainty, where one conjecture is perhaps as probable as another; for none of them can have any support, or derive any authority from the practical facts wherewith our experience has brought us acquainted.

In the discovery of hidden things and the investigation of hidden causes, stronger reasons are obtained from sure experiments and demonstrated arguments than from probable conjectures and the opinions of philosophical speculators of the common sort...

In the space of one hundred and seventy-six years the Lower Mississippi has shortened itself two hundred and forty-two miles. That is an average of a trifle over one mile and a third per year. Therefore, any calm person, who is not blind or idiotic, can see that in the old Oolitic Silurian Period, must a million years ago next November, the Lower Mississippi River was upward of one million three hundred thousand miles long, and stuck out over the Gulf of Mexico like a fishing-rod. And by the same token any person can see that seven hundred and forty-two years from now the Lower Mississippi will be only a mile and three-quarters long, and Cairo and New Orleans will have their streets joined together, and be plodding comfortably along under a single mayor and a mutual board of aldermen. There is something fascinating about science. One gets such wholesale returns of conjecture out of such a trifling investment of fact.

It may be observed of mathematicians that they only meddle with such things as are certain, passing by those that are doubtful and unknown. They profess not to know all things, neither do they affect to speak of all things. What they know to be true, and can make good by invincible arguments, that they publish and insert among their theorems. Of other things they are silent and pass no judgment at all, chusing [choosing] rather to acknowledge their ignorance, than affirm anything rashly. They affirm nothing among their arguments or assertions which is not most manifestly known and examined with utmost rigour, rejecting all probable conjectures and little witticisms. They submit nothing to authority, indulge no affection, detest subterfuges of words, and declare their sentiments, as in a Court of Judicature [Justice], without passion, without apology; knowing that their reasons, as Seneca testifies of them, are not brought to persuade, but to compel.

The entire annals of Observation probably do not elsewhere exhibit so extraordinary a verification of any theoretical conjecture adventured on by the human spirit!
[On the mathematical work by Urbain Le Verrier predicting the planet Neptune.]

The philosophical study of nature rises above the requirements of mere delineation, and does not consist in the sterile accumulation of isolated facts. The active and inquiring spirit of man may therefore be occasionally permitted to escape from the present into the domain of the past, to conjecture that which cannot yet be clearly determined, and thus to revel amid the ancient and ever-recurring myths of geology.

The rules of scientific investigation always require us, when we enter the domains of conjecture, to adopt that hypothesis by which the greatest number of known facts and phenomena may be reconciled.

The world has arisen in some way or another. How it originated is the great question, and Darwin's theory, like all other attempts, to explain the origin of life, is thus far merely conjectural. I believe he has not even made the best conjecture possible in the present state of our knowledge.

What certainty can there be in a Philosophy which consists in as many Hypotheses as there are Phaenomena to be explained. To explain all nature is too difficult a task for any one man or even for any one age. 'Tis much better to do a little with certainty, & leave the rest for others that come after you, than to explain all things by conjecture without making sure of any thing.

[Great scientists] are men of bold ideas, but highly critical of their own ideas: they try to find whether their ideas are right by trying first to find whether they are not perhaps wrong. They work with bold conjectures and severe attempts at refuting their own conjectures.