The term 'theory' has at least three main usages, and, though they all have similarities on the broad strokes, they are also quite distinct on the technical level. Let's begin with the general usage of the term.
In general usage, a theory is a collection of facts, definitions, rules, heuristics, and approved procedures, all applied in a specific area.For examples, we have the theory of nursing, the theory of aggressive play in chess openings, the theory of city planning. All of these will require certain facts, rules (constraints), procedures, definitions, and heuristics. Yet, none of these can be reduced to a purely logical system based on a set of axioms and deductions therefrom.
Next, we move to the more formal meaning of 'theory':
In formal usage, a theory is a collection of axioms (postulates), definitions, rules of deduction, and logical conclusions (theorems) drawn therefrom.The set of axioms to a theory are assumed to be self-consistent, and anyone is free to contest their self-consistency.
So, when Sherlock Holmes proposes a theory of how a crime was committed and by whom, he sets down certain assumptions (axioms) and draws certain logical conclusions (deductions) from them. In this example, we can see how a theory can be used to explain something.
In mathematics, theories, by convention, tend to cover broad categories of math. Examples are set theory, group theory, number theory, and ring theory. In each of these categories, we have a set of definitions, axioms, and theorems (logical deductions). Sure, in a given mathematical theory, one can use it to explain a certain fact, but, more often, a mathematician is concerned with discovering all the possible deductions (theorems) that can be proved from its set of axioms. For example: In group theory, one is often guided by the two questions: what are all the possible groups and what is the structure of each group? And, you can go one more step to ask: What are the possible relationships between different groups?
In science, things get a lot more complicated. First, a scientific theory is once again a formal theory, according to the last definition. A scientific theory need not be broad in scope. A scientific theory will very often be used to explain things. Second, a scientific theory, rather than a mathematical theory, is more likely to be burdened with hidden assumptions that will need to be recognized later on. Third, a scientific theory has a formal point of view that may be controversial. Fourth, a scientific theory must make empirically testable predictions and be accountable to those predictions. Fifth (and this is my structuring of the definition), a scientific theory has to undergo three levels of vetting before it is generally accepted. Now, I'll define three levels of scientific theory:
1) First, the theory is 'proffered' for specific immediate explanation or for later general consideration.
2) When the theory has been accepted by the majority of the experts in the
relevant field as more or less usable, it is said to be 'provisional'.
3) When the theory has been accepted by the majority of the experts in the
relevant field as reliable over its domain of applicability, it is said to be
'conventional' or 'accepted'.
I point out that the above definitions are not generally accepted. I introduced them to deal with the apparent ambiguity of the word 'theory'.
So, when Doctor Who is forced to hastily propose a theory of how a crime was committed and by whom, given a mere five seconds of brainstorming, he may proffer his best guess to those around him, to which he may receive a bunch of doubtful looks in return. If, in his own defense he should reply, "Well, it's only a theory," that would imply that theories themselves are what are at fault; that they are intrinsically weak, but they're not! The problem isn't in the nature of theory; it's in the requirement that a good theory must first be vetted properly -- and that takes time. What the good doctor should say instead is, "Well, it's only a hastily devised theory," which, although correct, would seem to be too much effort, even for clarity's sake.
Let me give an example of how a proffered (or nascent) theory is generally accepted in practice. In 1905, Einstein submitted a paper (to a prestigious journal of his day) called, 'On the Electrodynamics of Moving Bodies'. The theory he proposed (which later became what we now know as Special Relativity) adopted a different formal point of view than the basis on which electrodynamics was founded at that time, namely, on the assumption that light needed an invisible luminiferous ether for its propagation through space. Einstein's new formal point of view rendered the luminiferous (mechanical) ether superfluous. In other words, one can do electrodynamics without using the ether.
For many physicists, this was a welcome change of viewpoint. But for others, it was a tough pill to swallow. But over the next couple decades, the majority of physicists would adopt special relativity as the preferred mode of reasoning, and we can call this period the 'provisional' status of the theory. At some point, the theory became the tried-and-true way to think about electrodynamics and many other physical phenomena, until, now, the theory is the 'accepted' or 'conventional' theory to explain such phenomena (and much more), and it has been since the mid-1920s.
A scientific theory that has been properly vetted over decades, or even centuries, is not a weak or untrustworthy thing, especially when restricted to its domain of applicability. At this point, it is not 'only a theory'! Newton's theory of classical mechanics got man to the moon and back. It can also tell us how much of a load a front-end loader can safely endure without tipping over, and why a projectile follows a parabolic trajectory. The 150-year old theory of classical thermodynamics is used to warm your abode in winter and cool it in summer. These are very reliable theories!
However, theories aren't themselves laws, and are never promoted to laws. In the terminology of this essay, a provisional theory can be promoted to an 'accepted' theory, but never to a law. Why? Simply because a law is not a type of theory.
A theory must be founded on at least two logically independent axioms. So, what if one has only one axiom to work with? Say we proffer the axiom: What goes up, must come down. We can't found a theory on just that, but we can found an application of modus ponens on it:
(If something goes up, then it comes down) and (the ball goes up) implies (the ball comes down).
As we have just seen, much of the confusion about the nature of theories comes from the fact that there are different kinds of theories (conforming to different purposes and to different levels of rigor) and there are different standards of vetting on these kinds of theories.
Now, we must address the further complications to confusion because of how science textbooks often misrepresent the distinctions between theory, hypothesis, and law.
For our use in this essay, a hypothesis is the assertion of a simple proposition. (A proposition is a statement that is either true or false.)
A law of nature is an invariable relationship on the measurables of the physical realm. (This naive definition avoids a lot of unnecessary and subtle complications that don't help at the moment, though they would certainly be of interest to certain philosophers of science.)
The Theory of Special Relativity is not a law of physics, though it contains many laws of physics; one being that the measured speed of light in an inertial reference frame is a fixed value and is independent of the inertial frame in which it is measured. This law could well have started as a hypothesis and have been promoted officially to a law, which Einstein referred to as the Light Principle.
Theories are founded on a collection of hypotheses, which we have also called 'axioms' of the theory. Thus, any claim in science textbooks that, using the symbol '-->' to mean 'can be promoted to':
hypothesis --> theory --> lawis completely wrong and horribly confusing.
What we can say, however, is that, when appropriate:
1) hypothesis --> law, and
2) a proffered theory --> a provisional theory --> an accepted theory.
Thus, if you ever find yourself saying, "Well, it's only a theory," you are probably, at best, confusing the issue and, at worst, just plain wrong.