To explore this topic, programs/subprograms expressed in Java-like code are used rather than Turing machines. This is justified by the fact that Java programs are equivalent to Turing machines in terms of what functions can be computed and what languages can be decided/recognized.
We define a decision method to be a Java method that takes a string parameter and returns a boolean value in all cases (i.e., it never fails to halt). Examples:
/* Accepts { ax | x ∈ Σ* }
*/
boolean ax(String s)
{ return s.length()>0 && s.charAt(0) == 'a'; }
/* Accepts ∅
*/
boolean emptySet(String s) { return false; }
/* Accepts Σ*
*/
boolean SigmaStar(String s) { return true; }
|
Recall that a language L is said to be recursive (also called decidable) iff L = L(M) for some total TM (meaning one that halts on all inputs). Because Java is "Turing equivalent", the meaning would not change if we said instead that L is recursive/decidable iff L = L(m) for some Java decision method m.
Decision Method vs. Recognition Method:
If we relax the requirement of always halting, we get a
definition for recognition method. That is, a
recognition method is one that takes a string parameter
and returns a boolean value, but it need not satisfy the
property of halting on all possible inputs.
The language accepted/recognized by a recognition method
includes precisely those strings for which it returns true.
For strings not in that language, the method either returns
false or runs forever.
To state it in another way, a decision method is a recognition method that always halts (no matter what input is given to it), analogous to a total TM, which is a TM that always halts. So every decision method is also a recognition method, but not vice versa. (Just like every total TM is a TM, but not vice versa.)
Recall that a language L is said to be recursively enumerable (RE) (also called semi-decidable) iff L = L(M) for some (not necessarily total) TM. Because Java is "Turing equivalent", the meaning would not change if we said instead that L is recursively enumerable iff L = L(m) for some Java recognition method m.
Here is an example of a recognition method that is not a decision method:
/* Recognizes the language { anbncn }
*/
boolean anbncn1(String s) {
if (s.indexOf("ba") != -1) { return false; }
if (s.indexOf("ca") != -1) { return false; }
if (s.indexOf("cb") != -1) { return false; }
// Having verified that no "ba", "ca", or "cb" occurs in s, now see
// whether s meets further requirements to be accepted ...
String as = "", bs = "", cs = "";
// loop invariant: after k loop iterations, as+bs+cs == a^k b^k c^k
while (!s.equals(as + bs + cs)) {
as = as + 'a'; bs = bs + 'b'; cs = cs + 'c';
}
// If execution reaches this point, z.equals(as+bs+cs),
// and so s should be accepted.
return true;
// alternative loop:
//while (true) {
// String z = as + bs + cs;
// if (s.equals(z)) { return true; }
// as = as + 'a'; bs = bs + 'b'; cs = cs + 'c';
//}
}
|
This method returns true when given a string of the form anbncn. It returns false when given a string in which any of "ba", "ca", or "cb" occurs as a substring. Otherwise, it fails to halt.
The existence of this method demonstrates that the language { anbncn } is recursively enumerable (aka semi-decidable). This is no great accomplishment, because we already knew that this language is recursive/decidable. The point of showing Java method anbncn1() was to have an example of a recognition method that does not meet the more stringent requirement (of halting on all inputs) to qualify as a decision method.
A decision method for that language (whose existence demonstrates that the language is recursive/decidable) is as follows:
/* Decides the language { anbncn }
*/
boolean anbncn2(String s) {
String as = "", bs = "", cs = "";
String z = as + bs + cs;
while (z.length() <= s.length() && !z.equals(s))
as = as + 'a'; bs = bs + 'b'; cs = cs + 'c';
z = as + bs + cs;
}
return z.equals(s);
// alternative:
//while (true) {
// String z = as+bs+cs;
// if (z.length() > s.length()) { return false; }
// else if (z.equals(s)) { return true; }
// as = as + 'a'; bs = bs + 'b'; cs = cs + 'c';
//}
}
|
This method is horrible in terms of how much work it would do, but here we are not concerned with efficiency.
/* Interprets the first argument as the source code of a Java recognition
** method P and the second argument as a string to be fed to that method.
** If P is not syntactically valid, false is returned. Otherwise, the
** value returned is the same as that returned by the call P(x). (If
** that call would never result in a value being returned, the same is
** true of this method.)
*/
boolean run(String p, string x) { ... }
|
Examples of calling the run() method, within the context of a Java program:
// Assign to p the source code of method SigmaStar():
String p = "boolean SigmaStar(String s) {return true;}";
run(p,"abc"); // will return true
|
Because the call SigmaStar("abc") would return true, so would the call to run() above. More examples:
// Assign to p the source code of method ax():
String p = "boolean ax(String s) { " +
" return s.length() > 0 && s.charAt(0) == 'a'; }"
run(p,"ba"); // will return false (because "ba" does not begin with 'a')
run(p,"abba"); // will return true (because "abba" begins with 'a')
|
How about this one:
// Assign to p the source code of method anbncn1():
String p = "boolean anbncn1(String s) {" +
" if (s.indexOf("ba") != -1) { return false; }" +
" if (s.indexOf("ca") != -1) { return false; }" +
" if (s.indexOf("cb") != -1) { return false; }" +
" String as = \"\", bs = \"\", cs = \"\";" +
" while (true) {" +
" String z = as + bs + cs;" +
" if (s.equals(z)) { return true; }" +
" as = as + 'a'; bs = bs + 'b'; cs = cs + 'c';" +
" }" +
"}"
run(p, "abc"); // will return true
run(p, "abbc"); // will run forever
run(p, "abba"); // will return false
|
Technically, the run() method takes two String parameters, rather than only one, so it doesn't meet our definition of a recognition method. But we could modify run() so that it takes a single argument, intended to be of the form p#x, which it would split into the prefix up to the occurrence of '#' and the suffix following that occurrence. The prefix p would be taken to be the source code of a Java recognition method and the suffix x the string to be fed to that method. In other words, run() would take the input p#x and run method p (i.e., the method whose source code is p) upon input x. (This assumes that '#' occurs in neither p nor x; if that is not the case, we would choose some other character (or sequence of characters) to use as a delimiter between p and x.)
With this understanding of the kind of tweaks that are possible, we can relax our definitions of recognition and decision methods to allow for the possibility of them having multiple arguments. Under this relaxed definition, the two-argument version of run() is a recognition method (but not a decision method, because it is not guaranteed to halt).
The analogous language, in the realm of the Java model of computation, is that recognized by run():
In other words L(run) includes those (ordered pairs of) strings (p,x) such that x ∈ L(p). Now, L(run) is RE (recursively enumerable (aka semi-decidable)) because it is the language accepted by a recognition method (namely, run() itself).
Question: Is L(run) recursive/decidable?
That is, does there exist a decision method with this specification:
/* Returns true if the call run(p,x) yields true, but returns
** false otherwise (i.e., if run(p,x) either yields false or
** runs forever).
*/
boolean shortcut(String p, String x) { ... }
|
The specification says that, for all p and x:
For example, if p were the source code of the anbncn1() method, then shortcut(p,"abbc") would yield false, unlike run(p,"abbc"), which runs forever.
It would seem that shortcut() (like run()) must have the ability to simulate a given recognition method, but the extra capability of detecting when that method had entered an infinite loop (or of predicting such an occurrence).
It would be nice to have a debugging tool capable of determining, for a given program and input intending to be fed to that program, whether or not the program would get caught in an infinite loop. Alas, that is not possible, as we we now show!
The proof is "by contradiction". We begin by assuming that shortcut() exists and then go on to show that that assumption implies something that we know to be false. (The basis for proof by contradiction is that the boolean expression ¬x ⇒ false is equivalent to x = true. Thus, to show that x is true, it suffices to show that false is a consequence of the negation of x.)
Step 1: Use shortcut() to develop selfAccepting():
/* Returns true if recognition method p accepts its own source code;
** otherwise returns false.
*/
boolean selfAccepting(String p) { return shortcut(p,p); }
|
Step 2: Use selfAccepting() to develop nonSelfAccepting():
/* Returns true if recognition method p does not accept its own
** source code; otherwise returns false.
*/
boolean nonSelfAccepting(String p) { return !selfAccepting(p); }
|
We have (under the assumption that shortcut() exists) that nonSelfAccepting() is a decision method with L(nonSelfAccepting) being the set of (source codes of) recognition methods that do not accept themselves.
Because sigmaStar() accepts every input string, certainly it accepts its own source code. Which is to say that it is self-accepting. Thus, the call selfAccepting(p1) yields true. Which means that the call nonSelfAccepting(p1) yields false.
Now consider the string p2:
The method ax() accepts precisely those strings whose
first character is 'a'. Its own source code begins with 'b',
and so it does not accept itself.
Thus, the call selfAccepting(p2) yields false,
from which it follows that the call nonSelfAccepting(p2)
yields true.
End of digression.
Step 3: Now suppose that p is the source code of the method nonSelfAccepting() and consider what value would be returned by the call nonSelfAccepting(p). The figure below shows two different proofs that such a call would yield true if and only if it also returned false, which is a logical contradiction.
nonSelfAccepting(p) yields true
= < by known behavior of nonSelfAccepting() >
p is the source code of a method
that does not accept itself
= < p is the source code of nonSelfAccepting() >
nonSelfAccepting() does not accept itself
= < nonSelfAccepting() always halts;
definition of self-acceptance >
nonSelfAccepting() yields false when
applied to its own source code
= < p is the source code of nonSelfAccepting() >
nonSelfAccepting(p) yields false
|
nonSelfAccepting(p) yields false = < by known behavior of nonSelfAccepting() > p is the source code of a method that accepts itself = < p is the source code of nonSelfAccepting() > nonSelfAccepting() accepts itself = < definition of self-acceptance > nonSelfAccepting() yields true when applied to its own source code = < p is the source code of nonSelfAccepting() > nonSelfAccepting(p) yields true |
It follows that there can be no method whose behavior satisfies the specification of nonSelfAccepting(). But the existence of such a method is implied by the existence of a method whose behavior satisfies the specification of the shortcut() method, which is a decision method for the language L(run) (which is just the Java-centric form of the universal language Lu). It follows that the shortcut() method cannot exist, which means that the language L(run) is not recursive/decidable.
Summarizing, L(run) is recursively enumerable (RE, semi-decidable) but not recursive/decidable. Similarly for Lu.
The proof begins by assuming (contradictory to what is to be proved) that Lh is decidable, which would mean that the following decision method exists:
/* Returns true if the method whose source
** code is p halts when given x as input;
** otherwise returns false.
*/
boolean halts(String p, String x) { ... }
|
Define a narcissist to be a method that, given its own source code as input, runs forever. Making use of the halts() method, we can devise a recognition method for the language whose elements are the narcissists:
/* Given the source code of a Java method, returns
** true if that method is a narcissist and runs
** forever if it is not.
*/
boolean isNarcissist(String p) {
if (halts(p,p)) {
// p is not a narcissist, as it halts when applied to itself
while (true) { } // go into infinite loop!
}
else {
// p is a narcissist, as it fails to halt when applied to itself
return true;
}
}
|
Notice that the isNarcissist() method returns true when given (the source code of) a narcissist, but fails to halt when given (the source code of) a non-narcissist.
Let p be the source code of the isNarcissist() method, and consider what would happen if we made the call isNarcissist(p). The figure below shows two proofs that this call will yield true if and only if it would run forever, which is a logical contradiction.
isNarcissist(p) yields true = < code of isNarcissist() > halts(p,p) yields false = < specification of halts() > the method whose source code is p runs forever when given itself as input = < p is the source code of isNarcissist() > isNarcissist(p) runs forever |
isNarcissist(p) runs forever
= < code of isNarcissist() >
halts(p,p) yields true
= < specification of halts() >
the method whose source code is p
halts when given itself as input
= < p is the source code of isNarcissist() >
isNarcissist(p) halts
= < on any input, isNarcissist() either
returns true or runs forever >
isNarcissist(p) yields true
|
It follows that no method that behavies in accord with the specification of isNarcissist() can exist. But the existence of such a method is implied by the existence of the halts() method. Hence, that method cannot exist either.
/* Returns true if the method whose source code
** is p halts when fed input x. Otherwise fails
** to halt. Hence, this method recognizes Lh.
*/
boolean haltsRE(String p, String x) {
run(p,x);
return true;
}
|
We took great pains in showing that neither Lu nor Lh is decidable/recursive, from first principles, as it were. Another, typically easier, way to show that a language is undecidable (i.e., not recursive) is by way of algorithmic reduction.
Turing Reducibility
A reduction of a problem A to a problem B is,
broadly, a description of how an algorithm that solves B
can be used for the purpose of solving problem A.
Some authors use the notation A ≤t B to denote
that A is "Turing-reducible" to B.
Suppose that A ≤t B. Then the existence of an algorithmic solution for problem B implies the existence of an algorithmic solution for A. But that also means (by contraposition) that if there is no algorithmic solution for A then there cannot be an algorithmic solution for B, either.
Which is to say that to show that a language L is undecidable, it suffices to demonstration that L' ≤t L for some (already-known-to-be) undecidable language L'. Such a demonstration is typically has this form:
Example: Lempty is not recursive:
Consider the language
Assume that isMemberOfLempty() is a decision method for this language. Then we could devise a decision method for (the undecidable language) Lh, as follows:
/* Returns true if the method whose source
** code is p halts when given x as input;
** returns false otherwise.
*/
boolean halts(String p, String x)
String quotedP = "\"" + p + "\"";
String quotedX = "\"" + x + "\"";
String x2 =
"boolean f(String z) { " +
" run(" + quotedP + "," + quotedX + ");" +
" return true;" +
"}"
return !isMemberOfLempty(x2);
}
|
The "trick" here is that the value assigned to x2 is the source code of a method f() that has this behavior:
The following logic leads us to recognize that the existence of the method isMemberOfLempty would make the halts() method above a decision method for the language Lh. But we already knew that there can be no decision method for Lh. Hence, the isMemberofLempty() method cannot exist either.
<p,x> ∈ Lh = < behavior of the run() method > run(p,x) halts = < behavior of f() > any call to f() yields true = < x2 is the source code of f() > x2 is the source code of a recognition method that accepts all strings = < definition of Lempty > x2 ∉ Lempty = < assumed behavior of isMemberOfLempty() > isMemberOfLempty(x2) yields false = < return statement in halts() method > halts(p,x) yields true |
<p,x> ∉ Lh = < behavior of the run() method > run(p,x) runs forever = < behavior of f() > any call to f() runs forever = < x2 is the source code of f() > x2 is the source code of a recognition method that accepts no strings = < definition of Lempty > x2 ∈ Lempty = < assumed behavior of isMemberOfLempty() > isMemberOfLempty(x2) yields true = < return statement in halts() method > halts(p,x) yields false |
Another Example: Lreg is not recursive:
Consider the language
Assume that isMemberOfLreg() is a decision method for this language and recall that the method anbn() is a decision method that accepts the language {anbn}, which is not a regular language. Then we could devise a decision method for (the undecidable language) Lh, as follows:
/* Returns true if the method whose source
** code is p halts when given x as input;
** returns false otherwise.
*/
boolean halts(String p, String x)
String quotedP = "\"" + p + "\"";
String quotedX = "\"" + x + "\"";
String x2 =
"boolean f(String z) { " +
" run(" + quotedP + "," + quotedX + ");" +
" return anbn(z);" +
"}"
return !isMemberOfLreg(x2);
}
|
The "trick" here is that the value assigned to x2 is the source code of a method f() that has this behavior:
The following logic leads us to recognize that the existence of the method isMemberOfLreg would make the halts() method above a decision method for the language Lh. But we already knew that there can be no decision method for Lh. Hence, the isMemberofLreg() method cannot exist either.
<p,x> ∈ Lh
= < behavior of the run() method >
run(p,x) halts
= < behavior of f() >
the call f(z) yields true iff z ∈ {anbn}
= < x2 is the source code of f() >
x2 is the source code of a
method that decides {anbn}
= < definition of Lreg;
{anbn} is not regular >
x2 ∉ Lreg
= < assumed behavior of isMemberOfLreg() >
isMemberOfLreg(x2) yields false
= < return statement in halts() method >
halts(p,x) yields true
|
<p,x> ∉ Lh = < behavior of the run() method > run(p,x) runs forever = < behavior of f() > any call to f() runs forever = < x2 is the source code of f() > x2 is the source code of a method that recognizes the empty language, which is regular = < definition of Lreg > x2 ∈ Lreg = < assumed behavior of isMemberOfLreg() > isMemberOfLe(x2) yields true = < return statement in halts() method > halts(p,x) yields false |
Generalization: Rice's Theorem
So far, we have seen proofs that none among the languages
L(run) = { (p,x) | p is (the source code of) a recognition
method that returns true when fed x as input }
Lh = { (p,x) | p is (the source code of) a recognition
method that halts when fed x as input }
Lempty = { p | p is (the source code of) a method
that recognizes the empty language }
Lreg = { p | p is (the source code of) a method
that recognizes a regular language }
is a recursive/decidable language.
Rice's theorem is a sweeping generalization that gives us all of these results plus many more.
Rice's Theorem: For every nontrivial property α, the language
is not recursive/decidable
By a "trivial property" is meant any property that either all or no recursively enumerable languages possess. For example, L is accepted by some Turing machine (equivalently, by some Java recognition method) is true of all RE languages (by definition) and thus is a trivial property. The cardinality of L is uncountable is a property possessed by no RE languages and thus is trivial.
To state it in less technical terms, Rice's Theorem says that every nontrivial property of the RE languages is undecidable. In other words, for any "interesting" property that a language could possess, there can exist no algorithm that, given the source code of a Java recognition method, is guaranteed to tell us whether or not the language accepted by that recognition method possesses that property.
Webber's Slide 61 (among Chapter 18 slides) lists several languages that Rice's Theorem tells us are undecidable.
We have defined a recursively enumerable (RE) language as one that is recognized by some Java recognition method. The name thus seems to be a misnomer. (Why not call them the recursively "recognizable" languages?) Ah, but it turns out that languages recognized by Java methods are the same ones as are enumerated by language enumerators.
Turing's original paper viewed his "machines" as language enumerators rather than as language recognizers. A language enumerator M takes no input but simply generates a sequence of strings x0, x1, x2, ... possibly with duplicates (and, in the case of an infinite language, necessarily without end) The language that it enumerates is
As the definition implies, for x to be a member of L(M), it must appear among the first n strings to be generated, for some (finite) value of n.
Using a Java-centric point of view, here is an interface that models the concept of an enumerator, and a class that implements it:
public interface Enumerator {
String next();
}
|
public class A_Star implements Enumerator {
// instance variable
// -----------------
private String s;
// constructor
// -----------
public A_Star() { s = ""; }
// method of interest
// ------------------
public String next() {
result = s;
s = s + 'a';
return result;
}
}
|
The example shown, A_Star, enumerates the members of its language (i.e., { an | n≥0}) in increasing order by length —and without duplication— but there is no rule that says an enumerator must abide by those restrictions.
A more interesting example is an enumerator that generates all ordered pairs of natural numbers { (i,j) | i,j ≥ 0 }
public class NatPairsEnumerator implements Enumerator {
// instance variable
// -----------------
private int i, n;
// constructor
// -----------
public NatPairsEnumerator()
{ i = 0; n = 0; }
// method of interest
// ------------------
public String next() {
result = "(" + i + "," + (n-i) + ")";;
if (i == n) {
i = 0; n = n+1;
}
else {
i = i+1;
}
return result;
}
}
|
As an exercise, the reader is invited to devise a Java class that enumerates Σ*, where Σ = {a,b}. More interesting, and a little more challenging, would be to enumerate all sequences whose elements are the members of the set Nk = { 0, 1, 2, ..., k-1 }, for some fixed k > 0. In effect, this would be equivalent to an enumeration of an arbitrary alphabet Σ of cardinality k. (For k=2, that corresponds to an enumeration of all bit strings. For k=10, that corresponds to an enumeration of all (unsigned) decimal numerals.)
If we can create an instance of a Java enumerator class, it is easy to devise a method that returns the i-th string that is generated by that enumerator. As an example, suppose that SigmaStarEnum is such a class (and that, as its name suggests, it enumerates the language Σ* for some alphabet Σ).
/* Returns xi, where the strings generated by SigmaStarEnum
** are x0, x1, ...
** pre: i ≥ 0
*/
public static String SigmaStarIth(int i) {
Enumerator e = new SigmaStarEnum();
// Call next() i times to generate x0 ... xi-1
for (int j = 0; j != i; j++) {
e.next();
}
// generate and return xi
return e.next();
}
|
With this machinery in place, we are ready to prove that a language is recognized by some Java recognition method iff it is enumerated by some class that implements the Java Enumerator interface.
First we show implication from right to left. Assume the existence of a Java enumerator AEnumerate. Here is a Java method that recognizes the language that AEnumerate enumerates:
/* Recognizes L(AEnumerate)
*/
public static boolean aRecognize(String s) {
Enumerator e = new AEnumerate();
// Iterate until e generates s (which may be never)
while (!s.equals(e.next())) {
// do nothing
}
// Execution reaches here iff s is among
// the strings enumerated by e
return true;
}
|
Showing implication in the other direction is more tricky. Let aRecognize() be a Java recognition method whose language is a subset of Σ*, and let SigmaStarEnum be an enumerator that enumerates Σ*. A failed attempt to design an enumerator of L(aRecognize) is the following:
/* A failed attempt to devise an enumerator for the
** language recognized by the method aRecognize().
*/
public class BadAEnumerate {
// instance variable
Enumerator e = new SigmaStarEnum();
public String next() {
String s;
boolean keepLooking = true;
// Repeatedly call e.next() until it produces a
// string that is recognized by aRecognize()
while (keepLooking) {
s = e.next();
if (aRecognize(s)) {
keepLooking = false;
}
}
// s is recognized by aRecognize(), so generate it.
return s;
}
}
|
The trouble with this proposed solution is that aRecognize(), being only a recognition method (and not necessarily a decision method), may fail to halt when a string that it does not recognize is passed to it. If that were to happen before all the strings recognized by aRecognize() had been considered, the next() method in BadAEnumerate will fail to generate all the strings that it "needs to".
To fix this, we devise a modified version of our run() method (the Java equivalent of a universal Turing machine) that has an extra parameter whose purpose is to define a limit on how many steps of the target method should be simulated. The modified method is called runWithTimeLimit, which has this behavior:
The call runWithTimeLimit(p,x,n) (where p is interpreted to be the source code of a Java recognition method P()) returns true if the call P(x) returns true after n or fewer computational steps, and returns false otherwise. (Exactly how to count computational steps is a detail to be worked out.) Note that, regardless of its inputs, runWithTimeLimit() always halts.
It should not be hard to accept that the run() method can be augmented to keep track of how many simulation steps it has made, so that if that number reaches a given threshold, the simulation can be terminated and false returned.
Armed with a runWithTimeLimit() method, the NatPairsEnumerator class, and the SigmaStarIth() method (which depends upon the SigmaStarEnum enumerator), the following class shows that if aRecognize() is a recognition method, its language can be enumerated:
/* An enumerator for the language recognized by the
** Java method aRecognize().
*/
public class AEnumerate implements Enumerator {
// instance variable
Enumerator e = new NatPairsEnumerator();
String code = <source code of aRecognize()>
public String next() {
String s;
boolean keepLooking = true;
while (keepLooking) {
int (j,k) = e.next();
s = SigmaStarIth(j);
if (runWithTimeLimit(code, s, k)) {
// s is recognized by aRecognize() in k or fewer steps
keepLooking = false;
}
}
return s;
}
}
|
To prove that every string recognized by aRecognize() will eventually be the return value of the next() method in AEnumerate, suppose that s is such a string. Observe that there exists some j such that s = SigmaStarIth(j) (because the SigmaStarEnum enumerator generates every member of Σ* at least once). Second, there exists some k such that aRecognize() uses no more than k computational steps to recognize s, which means that the call runWithTimeLimit(code, s, k) results in true. Because the next() method of e generates every ordered pair of natural numbers, at some point there will come an iteration of AEnumerate's next() method during which the pair (j,k) will be generated by e. During that iteration, the call to runWithTimeLimit() will result in true, leading to s being returned by the next() method of AEnumerate, as required.
We can also be sure that AEnumerate's next() method never produces a string that is not recognized by aRecognize(), because any such string s is such that the call runWithTimeLimit(code, s, k) will yield false for every value of k.