BingChat and ChatGPT try problems in Symbolic Logic

The following is what I presented to both BingChat and ChatGPT.
[Note: I have corrected some of my own misspellings.]


Symbolic logic

Let True = 1 and False = 0

Let the following binary operators be
      | = logical or
      & = logical and
      > = logical implication.

Let the following unary operator be
      ~ = logical negation.

Let A, B, C, D be logical variables.

Initialization of variables:
Let A = True
      B = True
      C = False
      D = False

Evaluation order rule: When choosing between an adjacent unary and binary operator,
evaluate logical negation first (i.e., before evaluating any binary operator).

Now, find and report the values of the logical expressions:

1) ~((A&B)|C)

2) ((A|B)>~C)&D


My answers to the two expressions are False, False.

When I gave this problem to BingChat, it replied with:

Hello, this is Bing. I can help you with symbolic logic.(Happy Face)

To find the values of the logical expressions, we need to apply the rules of logic
and the given values of the variables.

~((A&B)|C) = ~((True&True)|False) //
substitute the values of A, B and C = ~((True)|False) //
apply logical and to True and True = ~(True) //
apply logical or to True and False = False //
apply logical negation to True

((A|B)>~C)&D = ((True|True)>~False)&False //
substitute the values of A, B, C and D = ((True)>~False)&False //
apply logical or to True and True = (True>True)&False //
apply logical negation to False = True&False //
apply logical implication to True and True = False //
apply logical and to True and False

Therefore, the values of the logical expressions are:

False
False


So, I have to tell the chatbots what the variables symbols are, what the logical
symbol's are, and what the variable values are. From there, it's up to each
chatbot to calculate the correct answers. By the way, I included the assignments
False = 0, True = 1, which might not be necessary.

My analysis of BingChat's performance is overall very good. It seems to know
what it's doing, although I had a bit of trouble following its explanation
of the steps involved. (I had the feeling that it was more 'thinking out loud'
than explaining.) Anyway, I give it 4.5 points out of 5.


When I gave this problem to ChatGPT, it replied with:

~((A&B)|C) can be evaluated as follows:

First, A&B is evaluated which is True & True = True.
Next, (A&B)|C is evaluated which is True | False = True.
Finally, ~((A&B)|C) is evaluated which is ~True = False.
Therefore, the value of the logical expression ~((A&B)|C) is False.

((A|B)>~C)&D can be evaluated as follows:
First, ~C is evaluated which is ~False = True.
Next, A|B is evaluated which is True | True = True.
Then, (A|B)>~C is evaluated which is True > True = False.
Finally, ((A|B)>~C)&D is evaluated which is False & False = False.
Therefore, the value of the logical expression ((A|B)>~C)&D is False.


Okay, ChatGPT seems to get the right answers and give a lucid explanation of the steps.
It applied the unary operation of negation first, which is correct. But would it have
done so if I had not instructed it to do so? I give it 5 out of 5 points.

Congratulations to both of them!

So, what should the follow up tests of their abilities be? Well, they should be tested on
more complicated expressions, and be tested on what they do with syntactically incorrect
expressions, such as
1) ((A|B)>~C)&&D,
2) ((A|B)>~)&D,
3) (()>~C)&D,
4) (A|B)>~C)&D,
and many more could be thought up.

1) is problematic because I have not defined a binary operator '&&'.
2) is problematic because there is no clear meaning to taking to the logical
negation of a null argument.
3) is problematic because a parenthesis pair ought to contain something.
And 4) is problematic because the number of left parentheses does not equal
the number of right parentheses.

Many years ago, when I had the time to write my own version of a symbolic logic evaluator,
by far, the hardest part of its coding was it syntax parser, which was much longer than
the evaluation module, and much trickier on the brain. And if my memory is correct, I
used recursion to do so.

If ChatGPT and BingChat ever hope to attain AGI (Artificial General Intelligence) status,
they should know how to write a complete syntax checker for a symbolic logic evaluator,
based on just the command: "Write a complete syntax checker for the symbolic-logic
evaluator you have received above." The syntax checker should throw out any syntactically
incorrect expression and explain to the user why it was rejected.