A sample is enough to assess
the quality of the harvest.

Computerized Audit Sampling

In Brief

A Simple Way to Achieve Reliable Results

Sampling methods used by auditors have evolved over the years. The trend now is to use less rigorous nonstatistical sampling to reduce cost. However, as demonstrated by an experiment conducted by the author, a nonstatistical approach to audit sampling can be substantially less capable of detecting material error than a statistical approach such as probability-proportional-to-size (PPS).

The use of qualitative analysis that documents the nature and cause of each misstatement found in a sample can mitigate some of the risk associated with sampling. The use of a statistical approach such as PPS can further reduce this risk, and, at the same time, permit the use of a smaller sample.

The downside of statistical sampling is its complexity when performed by hand. However, the use of an Excel-based software program called AuditAid that enables an auditor to use PPS and thereby gain greater efficiency and reliability from sampling operations in financial statement audits. The article demonstrates how the program works and includes exhibits showing its output. The program can be downloaded from the Journal's web site at http://www.cpaj.com/down.htm.

By Donald A. Schwartz

"The auditor has a responsibility to plan and perform the audit to obtain reasonable assurance about whether the financial statements are free of material misstatement, whether caused by error or fraud." (SAS No. 1, as amended by SAS 82, Consideration of Fraud in a Financial Statement Audit)

In the beginning, there was judgmental sampling:
"Considering the circumstances, in my judgment, a test of 30 of the larger items should give us adequate assurance as to whether the account is materially misstated."

Then, in the 1970s, statistical sampling's ability to provide quantifiable assurance attracted widespread attention among auditors:

"Based on these [statistical] results, we can be 95% confident that the amount of misstatement in the account does not exceed $50,000."

But by the early 1980s, when SAS No. 39 was published, it became apparent that the benefits of statistical sampling came at a high price--too high for most practitioners. Many were intimidated by statistical sampling's complex mathematical equations. And those who could master the math soon realized that unless the sampled population was highly stratified, the sample size computed under classical variables sampling theory was excessive--that is, much larger than was perceived necessary to obtain reasonable assurance that the account balance was not materially misstated.

Then, in 1983, the AICPA's Statistical Sampling Subcommittee prepared an audit guide called Audit Sampling that described, along with so-called classical models, a statistical model called probability-proportional-to-size (PPS). The audit guide (which remains in effect to this day) lists several advantages of PPS over classical variables sampling. Among these advantages is the "automatic" stratification built into the unique sample selection process. But if PPS has to be performed manually, it involves an adding machine procedure that is so labor intensive and error prone as to deter the conscientious audit planner. So, as a practical matter, only those large firms with internally developed software have been able to apply PPS procedures and take advantage of the smaller sample size.

Shortly thereafter, in the October and November 1984 issues of the Journal of Accountancy, Wade Gafford and Douglas Carmichael presented a "nuts and bolts" approach to audit sampling. Their model, which they refer to as formal nonstatistical sampling, makes use of PPS mathematical theory for the determination of sample size, but offers "less rigorous" (their characterization) ways to select the sample.

Gafford and Carmichael solved another problem that perplexed would-be users of statistical sampling. Statistical sampling requires that the auditor declare a specific confidence percentage, as in the phrase, "I need a sample that will provide me with 95% confidence that ..." Auditors are well aware that the higher their assessments of inherent and control risks, the more dependent they are upon related substantive tests, the more reliable the sample has to be. But how much is enough? 95%, 90%, 80%, 70%? Gafford and Carmichael respond to this dilemma by providing tables in which an auditor's qualitative assessments of inherent and control risks are assigned values called risk factors which, in turn, are associated with confidence percentages ranging from 95% (the most rigorous) down to 50%. Rather than trying to pick the "right" percentage, auditors can thus express their risk assessments in qualitative terms such as maximum, slightly below maximum, moderate, and low.

But even the Gafford/Carmichael procedure is complex. A simpler adaptation of their model is presented in Practitioners Publishing Company's Guide to Audits of Small Business. Its forms for calculating sample size and projecting the misstatement are easy to follow and can be completed in minutes. The downside is that its sample selection and its misstatement projection techniques are even less rigorous than the Gafford/Carmichael model, which, in turn, is less rigorous than standard PPS procedures.

The Experiment

Just as PPS-based models have been made easier to use, they have correspondingly become less rigorous. Which raises the question, just how reliable is this latest version of the PPS model?

To get the answer, a hypothetical population was set up consisting of 1,000 accounts receivable account numbers and amounts, with several simulated misstatements totaling $52,500. Based on relatively high inherent and internal control risk assessments, the latest PPS model determined that a sample size of 61 items would be needed to reduce sampling risk to an acceptable level (approximately seven percent). To test the validity of this sample size, more than 500 random samples (not PPS) of 61 items each were drawn from the hypothetical population. The startling result was that for more than 21% of the random samples, the actual misstatement was three times as large as that projected from the sample. If this had been an actual audit situation, the auditor would unknowingly have been subjected to a 21% risk of accepting a materially misstated account balance--despite having stipulated that the sample should ensure a risk of no greater than seven percent.

The reason for this disparity is that unbiased selection (random, haphazard, or systematic) from an unstratified or even slightly stratified population can be substantially less capable of detecting material error than the PPS selection process. How much less capable depends upon how the misstatement is distributed throughout the population. For example, if an astute fraudster were to create just a handful of fictitious accounts receivable balances in amounts just below the individually significant threshold, the auditor who uses unbiased selection might be taking an 80% or 90% chance of finding no misstatement at all.

Caveats on PPS

The Audit Guide on Audit Sampling suggests that PPS sampling may be especially useful in the audit of accounts receivable and inventory. However, it is not appropriate for accounts receivable if there are a large number of unapplied credits, or for inventory where the auditor anticipates a significant number of audit differences, or where the detection of an understated balance is an important consideration.

Moreover, PPS evaluation technique is so sensitive to any errors found in the sample that it tends to overstate the allowance for sampling risk and thereby project a potential misstatement that could be two or three times the actual misstatement. The reason PPS tends to exaggerate its projection of misstatement is that it does not simply extrapolate the total error found in the sample. Instead, it looks at each erroneous item individually, and projects a misstatement amount proportional to that item's percentage of error rather than its amount of error. Thus, an item with a $100 book value but an audit value of $10 is considered 90% misstated and results in the same projected misstatement as a $1,000 item 90% misstated. Though this appears illogical, under the PPS selection method, the $1,000 item has ten times more chance of being selected for audit than does the $100 item. So when errors are found among the relatively few small items that have been selected, they are given proportionately more weight. But in so doing, PPS subjects the auditor to a high risk of incorrect rejection, that is, the risk of rejecting an account balance that is not materially misstated. To put it bluntly, PPS is prone to false alarms.

Qualitative Analysis

PPS's job is only to warn us of a possible fire, not to assess the extent of the fire or estimate the damage. This requires classical forms of statistical sampling and extracts the price of a much larger sample. The auditor's response to the alarm is essentially the same regardless of the degree by which PPS's projected potential misstatement exceeds tolerable misstatement (assuming the excess is more than trivial).

SAS No. 39 stresses the importance of qualitative as well as quantitative analysis. The auditor should identify and document the nature and cause of each misstatement found in the sample. It takes finding only one misstatement of a particular type for the auditor to become aware that that kind of misstatement is occurring, at which point the auditor can apply additional procedures to determine the extent of misstatements of that type. One misstatement may indicate a breakdown in a control procedure that suggests other errors of a similar nature, and might in fact have implications elsewhere in the audit. A second misstated item might clue the auditor to an inappropriate accounting principle that probably affects all similar transactions. By working with the client to identify and correct other similar errors, the potential misstatement might be reduced to an acceptable level. If not, other kinds of tests that serve the same audit objectives, such as appropriate analytical procedures, may provide the additional evidence needed to support the corrected book value of the account. Of course, if the possibility of fraud is indicated, further effort and more careful consideration are required.

Why Statistical Sampling?

Some practitioners question the usefulness of statistical sampling at all, since it certainly does not eliminate, or even reduce, the need for professional judgment. The auditor must still decide on a tolerable misstatement amount, assess related inherent and control risks, and respond appropriately to test results. But once these judgments have been made, statistical sampling does offer several important advantages. In their book, A Practitioner's Guide to Audit Sampling (John Wiley & Sons, 1998), Dan Guy, Douglas Carmichael, and Ray Whittington point to the practical advantages reported by auditors using statistical sampling:

* Less likelihood of over- or
underauditing

* Audit work is more objective and defensible

* Better workpaper documentation

* Greater confidence in the audit opinion

And because the PPS approach to statistical sampling enables the smallest possible sample to achieve the targeted low risk of accepting a materially misstated account balance, it is the most cost effective of the sampling alternatives.

Enter the Desktop Computer and
Its Software ...

Now, with the availability of inexpensive, user-friendly computers and software, the reliability and efficiency of the standard form of PPS statistical sampling can be easily achieved. The following scenario illustrates this computerized approach to audit sampling.

Katherine Smith, CPA, has been engaged by Media Enterprises to perform a financial statement audit of Tri-City Publishing Co., a family-owned newspaper Media is planning to acquire. In planning the audit of the $1,650,000 accounts receivable balance, Smith finds there are approximately 950 accounts with positive balances. Of these, a dozen are very large and comprise over one-third the total balance, but the remaining balance is made up of relatively small amounts. Though Smith has never used statistical sampling, she feels that under these circumstances (first-time audit, Tri-City's motivation to show maximum profits, and Media's reliance on Smith's opinion), a statistical approach would more likely provide the appropriate level of assurance and more defensible workpaper documentation than would a purely judgmental approach.

Importing Client Data

Smith's initial step was to ask the client to save its accounts receivable trial balance report onto a diskette as a text file. That gave her a diskette file containing account number, customer name, and amount as a separate line of text for each of the 1,000 customers. Smith then opened the text file in Microsoft Excel. Upon opening the file, Excel detected that the file was in text rather than spreadsheet format, and automatically applied its Text Import Wizard to place the account number, name, and amount for each customer into three separate columns of an Excel worksheet. Smith then copied these three columns of data into the data area of an Excel-based program called Audit-
Aid. She then used AuditAid menus to perform the sampling process.

The Sampling Process

The process for the substantive testing of a relatively large population includes the following steps.

Decide on the individually significant item amount to be audited 100%. Then, decide if the audit of all individually significant items (called key items for short) can provide reasonable assurance that the account is not materially misstated, or if the remaining items must be sampled.

To see an overview of the population, Smith clicked on an AuditAid worksheet tab called profile, in which customer balances were grouped into size categories (see Exhibit 1). From this analysis, Smith could tell at a glance that a small number of very large balances comprised approximately 33% of the total dollar amount, and that it would take several hundred more items to make up the 60­70% needed to avoid testing smaller balances. Smith concluded it would be necessary to test a sample of the remaining items.

Since Smith had not previously decided on a materiality threshold amount, she used the optional materiality worksheet to obtain suggested rule-of-thumb amounts for planning materiality, tolerable misstatement, and individually significant key item amounts.

Decide on an appropriate sample size. To determine an appropriate sample size, Smith went to the size worksheet, in which she entered the tolerable misstatement amount from the previous step. After reading the related help screens for guidance, she entered her assessment of the following risk factors (Exhibit 2):

* Reliance on this auditing procedure (positive confirmations in this case). Since some additional substantive testing would be performed by means of analytical procedures, Smith assessed her reliance on the receivable confirmations as slightly below maximum.

* Inherent risk. Smith's assessment of inherent risk was based on this being a first-time audit, the pending acquisition of the company, and other such considerations. She again chose slightly below maximum.

* Control risk. Smith's assessment of control risk, moderate, was based on her test of related internal controls.

Based on Smith's tolerable misstatement amount and risk assessments, the program suggested an allowable sampling risk of 12.5%. Smith entered this risk percentage, which the program then used to calculate a required sample size of 50 items. In calculating the appropriate sample size, the program used PPS statistical formulas, under the presumption that PPS techniques would likewise be used for sample selection, and also for projecting the likely misstatement and potential misstatement in the population.

Select and list for audit the key items and selected sample items. From the PPS menu, Smith chose select key items and sample items. The program extracted the individually significant or key items, and then selected the 50 sample items, using PPS selection procedure. Smith printed a list of the selected customer account numbers, names, and amounts, and asked the client to provide confirmation request letters for these customers.

Audit the key items and selected sample items. Smith examined the replies, investigated differences, and summarized her findings in the Excel spreadsheet called workpaper for test results.

Donald A. Schwartz, JD, CPA, is an associate professor of accounting at National University, San Diego, Calif.