The case for a disclosed exam. (certified public accountant examinations)by Pustorino, Anthony R.
I recognize that AICPA's Board of Examiners has decided to make the exam nondisclosed effective with the 1996 exam and that NASBA Board of Directors has given its support to the change. However, I believe that accountants may find it interesting to review the events that may have given rise to the change and what the profession may have lost in the process.
The genesis of this drive not to publish the exam in part stems from the discovery that certain exam candidates in Texas who participated in a computerized software coach course received higher than normal pausing rates in the exam sections in Theory, Practice and Law. The NASBA CPA Examination Review Board, of which I became a member after its deliberations on the disclosed exam issue, in reviewing the Texas situation concluded that "test score pollution" was at work, i.e., those "... practices which boost ... test scores without a corresponding change in the candidate's mastery of the content domain." From a broader perspective, the Review Board asserted that "candidates are focusing too much on test taking and test items at the expense of acquiring the knowledge and skills needed for the practice of public accounting." How does the Review Board know that? I am unaware of any empirical evidence that supports such an assertion. There is greater evidence to support the view that candidates who have studied for and passed the CPA exam, albeit with the help of a coach course, have at least acquired the entry level knowledge for the practice of public accountancy.
For many years, the profession has lamented the low passing rates on the exam. Now, when finally a learning technique seems to get the passing rate to rise, there is a decision to make the exam text unavailable, presumably to get the passing rate down again. Why is it so difficult to accept a postulate that maybe a better teachng process has been used in Texas and that in the process of reviewing past exam questions the candidates actually did learn something about accounting beyond how to pass the CPA exam? Many studies have confirmed the positive correlation between advanced academic degrees and the passing rate on the exam. NASBA's 1991 edition of the CPA Candidate Performance on the Uniform CPA Examination in May 1990 reports that 30.4% of first- time candidates with advanced degrees passed all subjects taken, while only 18.2% of those with baccalaureate degrees passed all subjects taken. Florida's experience confirms these results. Prior to Florida's passage of its 150-hour law in 1983, the passing rate for its candidates was approximately 14%. Since passage of the law, the rate has jumped to 32%. Clearly, more, better, and perhaps different education positively affects the passing rate. Why shouldn't the Texas candidates, all of whom were in a graduate program with high entry requirements, taught by good instructors who required proof of homework, not have better passing rates? Good education should be expected to do that.
Distracting to Seniors?
Another argument often advanced for not publishing the exam is to force senior accounting majors to concentrate on studying to pass their college course work rather than on studying to pass the CPA exam. I find the premise for this argument difficult to accept. Why would rational, senior accounting majors imperil their impending graduation by not studying to pass their course work first? Students know they have to pass their college courses before they can graduate in order to be eligible to sit for the exam. Furthermore, since job opportunities are usually directly linked to grades, as well as admittance to graduate schools, it is even less likely that seniors would jeopardize graduation by directing their study efforts to the CPA exam instead of to their college studies.
A corollary to this argument is the Review Board's assumption that there are some nefarious professors who only teach their students facts, not concepts, and who purposely omit the discussion of emerging issues of the profession. Instead, it is asserted they are teaching students only the relevant information needed to pass the exam. What evidence is there that educators are not teaching concepts and their applications? No professor I know of purposely shapes any of his or her courses as a CPA coach course.
By some circular reasoning the Review Board asserts that professors teach facts because the exam contains test items that are fact-specific. Wouldn't better questions in the exam itself go a long way toward addressing this situation if professors are really "exam driven?"
Let us also examine an underlying premise inherent in the proposed change--that the content of the exam should not direct accounting education. I certainly agree with that. But if the exam is based on surveys of what CPAs do, i.e., practice analyses, there must be some connection between practice and exam content; otherwise the surveys make little sense. If, then, the exam is truly "practice-driven," and a professor did indeed, "teach to the exam," wouldn't he also be teaching to practice? What would be so wrong that if somewhere, someone was actualy doing that?
Essay Questions Should Not Be
Wouldn't a more meaningful concentration on essay questions obviate the need for not publishing? One could draw a corollary between the effort to make it a non-published exam and the switch to predominantly objective-type questions in the future.
The proponents of not publishing the exam base much of their arguments on recommendations from psychometricians. We should remember that it was the psychometrician who originally recommended an all-objective exam, which was resoundingly rejected by NASBA some time ago. The wiser heads of NASBA believed the psychometricians to be wrong then. Why do they believe they are right now?
It is also in the public interest to keep the exam open. In the era of the impending 150-hour education requirement--which many believe will impact most severely on minorities--is it not likely that a non- published CPA exam will be perceived by some as simply one more obstacle put in the path of minority groups to inhibit their entry into the profession? When an exam is open, everyone can see it and everyone can make a judgment as to its fairness, objectivity, and lack of bias for, or against, any particular group. We peril that judgment and plant seeds of doubt in the public's mind concerning the exam's integrity if it is no longer disclosed.
Isn't it also naive to believe that the content of a non-published exam will not become public knowledge soon after it is given? In spite of the sanctions in the proposal attached to disclosing content, one's imagination need not be stretched too far to envision some who will take the exam for the sole purpose of remembering the questions so as to be able to divulge them afterwards to interested parties. I believe that the proposal will inadvertently sow the seeds of a fertile black market for the purchase and sale of past exam questions.
Without impugning the integrity of the many fine and honorable CPA coach courses that exist, we can be sure that some other resourceful courses will, one way or another, quickly gain knowledge of the questions asked right after the exam is over and subtly convey that information to the next batch of candidates. This will likely prompt many candidates to take such courses in order to gain insight to the type of questions asked. Those candidates who do not take such courses will be at a disadvantage. In one mighty and effective stroke, therefore, a nondisclosed exam will take away the level playing field for all candidates.
In any case, the laws of most states allow candidates to see their exams if they have failed, and proctors, administrators, and state board observers also have access to the exam while it is being given. Candidates who have taken the exam will talk to their friends, and others, and no doubt divulge the contents of the exam as a matter of course. What administrative structure is envisioned by the proponents of a nondisclosed exam to preclude--or punish--such expected behavior from so many different parties? And at what cost? I think it is ingenuous to believe that everyone who has seen the exam will purge its contents from his memory.
I don't believe the Texas phenomenon should cause us to sound the alarm, circle our wagons, and go under cover with a nondisclosed exam. It seems to me that the Texas experience merely confirms the obvious-- that better candidates, better prepared, do better on the exam. Nothing more. In the era oif 1,500-page text books on intermediate accounting and auditing, 100 plus SFASs, with many more in the pipeline, tax laws which change annually, and SASs issued in batches, there would seem to be no dearth of material available for the texts of all future, widely published CPA exams. Omega
Anthony R. Pustorino, CPA, is Chairman of the New York State Board of Public Accountancy, Member of the AICPA Council, Member of NASBA CPA Examination Review Board, Assistant Chairman of the Accounting Department at Pace University, and retired President/Shareholder of Pustorino, Puglisi and Co., P.C., CPAs.
The CPA Journal is broadly recognized as an outstanding, technical-refereed publication aimed at public practitioners, management, educators, and other accounting professionals. It is edited by CPAs for CPAs. Our goal is to provide CPAs and other accounting professionals with the information and news to enable them to be successful accountants, managers, and executives in today's practice environments.
©2009 The New York State Society of CPAs. Legal Notices
Visit the new cpajournal.com.