From: Charles Greenlaw <cgreenlaw(--nospam--at)speedlink.com>
Date: Fri, 25 Feb 2000 00:55:33 -0800
Bill, I'm glad your recent experiences with the SE Exam scoring have been
positive, and that both the the process and spirit seemed appropriate.
I can say the very same about every aspect of my role in the 1981 SE Exam,
except for what happened with the colleague's defective problem. The
practice then was to have each committee person, under clear guidance from
the local chairman, write a problem. Another person worked the problem at
home and in the next meeting report on how it went. The others also acted as
critics, and helped resolve any snags that were discovered. The other
fellow's retaining wall problem that I worked had inconsistent "givens" that
precluded a single statically determinate solution; three equally valid ones
were possible. Its author could not see this, and he felt there was only one
valid approach. The rest of the local committee, including the chairman,
agreed there were three different answers
The chairman did not press for corrections, but said he would have the
statewide committee verify the defect and change the problem before the exam
was printed. They botched the change and cured nothing. Then the problem's
author, who in accordance with the process back then graded his own problem
alone over a period of several months, AFTER clearing his grading plan with
the chairman graded his problem to his original intentions. Batches of exams
were exchanged among us for grading, and each time I saw what he was doing,
I alerted the chairman. That person assured me he would deal with it, but in
fact did nothing to change how the man was grading his problem. As a result
many good solutions were denied most of the credit available. Some examinees
checked their solutions using a different pair of static conditions, and you
should have seen the agonized comments on their papers about getting a
different result they could not reconcile. They deserved double credit, not
the one-third credit they were given.
The problem-fixing process was not defective, but the chairman's willingness
to uphold it for its intended purpose was a complete failure. He wimped out,
and betrayed the trust examinees deserve to have in the examiners. He stayed
on the Exam committee many more years, as did the problem writer who wasn't
as half as competent as the candidates he was testing.
Yet if I had test-worked someone else's problem instead, I probably would
never have noticed the bad one, and never believed the exam as given lacked
integrity and cheated the examinees. I submit that despite new grading
session procedures, the same failure possibility still exists: most
problems are satisfactory to those who are involved with them, as you have
experienced, but some can be botched and go unchallenged from within, or
tacitly go uncorrected to save face and to avoid discord within the
committee. Who that counts is ever to know?
A change for the worse is modern reliance on writing problems to use one
method only (of two or more) within a particular edition of a certain code,
such that the problem becomes one of pedantic code compliance. In 1981, we
were admonished not to tailor a problem to any particular code, on grounds
it is unfair to people from other regions and tests a factor that's not of
great interest and expires with the next code edition.
Another adverse change is use of multiple choice answer format for creative
solution problems, instead of only for factual knowledge questions that have
no possible alternative answers. Former SE Board member Hoi Wong worked on
SE exams for many years, and told me that the committee members hated to
write multiple choice questions, but the PE Board demands them.
I am aware that many changes in the last few years have been made that are
obviously responsive to lawsuit allegations. The PE Board belatedly assigned
both of their excellent CE staffers to exam administration. One's husband
and father-in-law are private practice SE's. It's fine that things are
improved after a rude awakening. But many other changes the PE Board made in
response have the clear purpose of making their inside doings more
impenetrable, secret, and unreviewable. That only caters to recurring human
nature factors that allow good systems to be corrupted from within all over
Charles O. Greenlaw SE
At 04:54 PM 02/24/2000 -0800, you wrote:
>I don't know whether California exam practice has changed since you
>participated or what.
>Although I had heard of rumors, prior to my participation, of the type of
>situations such as you described, I was pleasantly surprised to find that
>the instructions given were geared to allowing any and all reasonable
>After one of the sessions, one author I talked with was deeply disturbed
>that his problem was way too difficult and set the bar too high. He said he
>learned that it was more important to test fundamentals than detailed
>understanding of code provisions and that would impact any future problems
>he might author.
>Bill Cain, S.E.