8-64) LO 1, 3, 4, 5
Locate and read the article listed below and answer the following questions.
Hall, T. W., J. E. Hunton, and B. J. Pierce. 2002. Sampling practices of auditors in public accounting, industry and govern- ment. Accounting Horizons 16 (2): 125–136.
http://www.aaajournals.org/doi/pdf/10.2308/acch.2002.16.2.125
I need better answers then the ones provided by the textbook solutions.
a.) What is the issue being addressed in the paper?
b.) What are the findings of the paper?
c.) Why is this paper important to auditors, and what are the implications of this paper for the auditing profession?
d.) Describe the research methodology used as a basis for the conclusions.
e) Describe any limitations of the research.
Here is the article copy and pasted below:
Sampling Practices of Auditors in Public Accounting, Industry, and Government Thomas W. Hall, James E. Hunton, and Bethane Jo Pierce Thomas W. Hall is a Professor at The University of Texas at Arlington, James E. Hunton is a Professor at Bentley College, and Bethane Jo Pierce is an Associate Professor at The University of Texas at Arlington. SYNOPSIS: Although audit sampling is a common procedure, relatively little is known about the sampling practices of auditors in public accounting, industry, and government. This study surveyed practicing auditors to determine how they: (1) planned sample sizes, (2) selected sample items, and (3) evaluated sample outcomes. Respondents also provided data on the training received, debiasing techniques employed when using nonstatistical (judgmental) methods, and literature sources relied on to provide guidance regarding sampling matters. Respondents in all areas of practice reported that a majority of audit sampling applications rely on nonstatistical methods for sample planning, selection, and evaluation. Despite the heavy reliance on nonstatistical methods, less than 10 percent of respondents reported receiving training in debiasing techniques, and no respondents reported using these techniques. Among statistical methods dollar-unit sampling is the most frequently employed technique. All respondents reported reliance on employer guidelines, and most reported reliance on sampling standards promulgated by the American Institute of Certified Public Accountants. Keywords: sampling practices; audit sampling. INTRODUCTION Although audit sampling is a common procedure, relatively little is known about the sampling practices of auditors in public accounting, industry, and government. With the exception of Hitzig’s (1995) survey of public accounting firms in New York, most published studies are 20 years old and focus on the use of statistical methods by auditors in public accounting (see McRae 1982; Ross et al. 1981; Bedingfield 1975). One limitation common to all prior studies is that their survey instruments asked only if auditors used certain sampling techniques, but did not inquire about the frequency of use for each alternative method. To provide a more complete understanding of sampling practices this study surveyed auditors in public accounting, industry, and government.1 Issues investigated include the relative frequency of use of alternative methods for sample size planning, sample selection, and sample evaluation. We also collected data on the training received by auditors, literature sources relied on for guidance, and the use of debiasing techniques when employing nonstatistical (judgmental) methods. Survey respondents in public accounting, industry, and government reported similar sampling practices. Most audit sampling applications rely on nonstatistical methods for sample planning, selection, and evaluation. In spite of the heavy reliance on nonstatistical methods, less than 10 percent of respondents reported receiving training in debiasing techniques, and no respondents reported using these techniques. Among statistical methods, dollar-unit sampling is the most frequently employed technique. All respondents reported reliance on employer guidelines, and most reported reliance on sampling standards promulgated by the American Institute of Certified Public Accountants. These findings enable audit practitioners to benchmark their sampling practices against current norms of the profession. Further, in federal court proceedings the decision to accept sample evidence hinges on whether the facts or data are “of a type reasonably relied on by experts in the particular field in forming opinions or inferences upon the subject” (Federal Judicial Center 1994, 227). By documenting the current state of sampling practice this study provides potentially useful information for auditors and their attorneys, should an audit sample be challenged in court. Audit educators may use these findings when planning their course content, while students can use the findings to understand extant practice. METHODOLOGY Survey Instrument We developed the survey instrument to obtain auditors’ opinions and estimates regarding sampling practices they employed in the six months before they received the instrument. Approximately 20 experienced auditors, including partners, managers, and seniors from two national CPA firms and the chief auditors of several corporations reviewed the survey instrument prior to its use. After revising the instrument to incorporate the suggestions of these practicing auditors, we mailed the survey with an accompanying cover letter and return envelope in mid-1997. Approximately four weeks after the initial mailing we mailed second requests. The first three sections of the survey instrument asked how auditors: (1) determine sample sizes, (2) select samples, and (3) evaluate their samples. To gather information about sample size determination respondents considered all sampling applications completed over the past six months and then reported estimated percentages of all applications that relied on formal statistical vs. nonstatistical (judgmental) methods to set sample sizes. To determine how auditors select their samples we provided respondents a list (with definitions) of: (1) four statistical selection techniques (simple random, stratified, systematic/random start, and dollar-unit) and (2) four nonstatistical selection techniques (haphazard, block, systematic/judgment start, and directed). After reviewing these lists respondents reported estimates of the percentages of all sampling applications that relied on each technique. Finally, to determine how auditors evaluate their samples, respondents considered all sampling applications completed over the past six months and reported estimated percentages of applications that relied on formal statistical evaluations vs. nonstatistical (judgmental) evaluations. The survey instrument defined formal statistical evaluations as those that relied on statistical formulas, or tables or software based on statistical formulas. Nonstatistical evaluations included those that relied on the auditor’s judgment to form an opinion about the underlying population.2 Section four asked respondents to indicate the primary sources they consult for guidance regarding sampling and evaluation procedures. Respondents chose from a list of six sources: (1) American Institute of Certified Public Accountants (AICPA) auditing standards—AU Section 350, (2) the AICPA Audit Sampling Guide, (3) Institute of Internal Auditors (IIA) standards, (4) textbooks and monographs, (5) employer standards, and (6) other sources (fill in the blank). The fifth and sixth sections asked about sampling training received by respondents. Section five asked respondents to report the extent—none, minor, substantial3—of college-level training received in statistical sampling and evaluation. Respondents also provided an estimate of the hours of post-college training received in statistical sampling and evaluation techniques. Identical questions about training in nonstatistical techniques appeared in section six of the instrument. In addition, because several published studies document selection biases (see, for example, Hall et al. 2001; Yates 1937) and evaluation biases (see, for example, Ponemon and Wendell 1995; Kinney and Uecker 1977a, 1977b) inherent in nonstatistical methods, the survey instrument asked respondents to report on the extent of training received in bias avoidance. Sections seven and eight requested information about the perceived seriousness of selection biases when using nonstatistical selection procedures and the procedures employed to mitigate these selection biases. In section seven, respondents indicated on a seven-point scale (1 = none, 4 = some, 7 = great) the extent to which certain physical characteristics of population elements—color, size, and physical location—influence auditors’ nonstatistical selections. The survey instrument identified these particular physical characteristics because published research indicates they do influence nonstatistical sample selections (see Hall et al. 2001; Hall et al. 2000) and because AICPA literature states that these characteristics should not influence sample selections (AICPA 1995, §8220.05). Section eight listed four approaches to deal with nonstatistical selection biases: (1) do nothing, (2) increase sample size, (3) stratify the sample, and (4) other (fill in the blank), and asked respondents to indicate which approach they use in practice. Finally, sections nine and ten requested selected demographic information. Section nine asked respondents to estimate the number of samples selected and evaluated in the past six months. We asked this question to ascertain whether respondents had recent field experience in audit sampling. Section ten inquired about subjects’ professional qualifications (CPA, CMA, CIA), practice area (public accounting, industry, government, other), field of expertise (accounting, audit, tax, consulting, other), and position (entry-level/staff, senior/supervisor/manager, partner/vice president, other). Participants To obtain responses from many segments of the audit field, we mailed 600 instruments to randomly selected offices of auditors working in public accounting, industry, and federal and state government. We addressed the surveys to either the “partner in charge” of CPA firm offices or the “chief auditor” for corporations and governmental entities.4 The cover letter explained the nature of the survey, asked for the recipient’s cooperation, and requested that he or she forward the instrument to an individual in the organization who had selected and evaluated samples as part of his/her routine duties within the past six months. We received a total of 223 usable responses for an overall response rate of 37 percent. The response rate was highest for auditors working in government (50 percent), followed by auditors in public accounting (36 percent), and auditors in industry (32 percent). All respondents reported that auditing was their primary job responsibility and that their duties within the past six months included the selection and evaluation of audit samples. Overall, respondents reported an average of 2.6 years of experience in positions that require the routine selection and evaluation of samples. Approximately one-half of the respondents held entry-level/staff positions while the other one-half held positions at the senior, supervisor, or manager levels. Fifty-three percent of respondents identified themselves as CPAs, 9 percent identified themselves as CMAs, and 5 percent held the CIA designation. RESULTS Planning, Selecting, and Evaluating Samples Table 1 reports the methods used by respondents to plan sample sizes, select samples, and evaluate sample results. From Panel A we see a key finding of the survey—on average, 15 percent of all sampling applications rely on formal statistical methods to plan sample sizes, while 85 percent of all applications rely on nonstatistical methods. We tested for differences in the responses of auditors in public accounting, industry, and government using a one-way analysis of variance (ANOVA). Like the common ttest, the ANOVA procedure tests for differences in means, but is capable of comparing more than two means at the same time. Test results yielded an insignificant5 p-value (.1606) and indicate a similar degree of reliance on nonstatistical planning procedures across all three areas of audit practice.
Panel B of Table 1 describes the methods respondents use to select audit samples. Consistent with prior data regarding determination of sample sizes, respondents indicated they select most samples (85 percent) using nonstatistical procedures. The most common method of sample selection is haphazard sampling at 74 percent of all applications, followed by dollar-unit sampling at 12 percent of all applications. A one-way ANOVA found no statistically significant differences between the responses of auditors in public accounting, industry, or government (p-value = .1606). In Panel C of Table 1 respondents reported evaluating approximately 36 percent of all sampling applications with formal statistical methods and 64 percent with nonstatistical methods. A one-way ANOVA found no evidence of statistically significant differences in the responses of auditors in public accounting, industry, or government (p-value = .0637). We note that auditors selected about 15 percent of their samples using statistical selection techniques, but evaluated approximately 36 percent of their samples with statistical evaluation methods. Thus, about 21 percent of auditors’ samples are improperly evaluated with statistical methods. To test whether this practice differed across various segments of the audit profession, we calculated for each respondent a measure representing the difference between: (1) the percentage of samples selected using statistical techniques, and (2) the percentage of samples evaluated using statistical techniques. We then used an ANOVA procedure to test whether this measure varied across segments of the audit profession. Results suggest that this practice does not vary based on auditor rank, years of audit experience, or the holding of a professional credential. However, the ANOVA test for the practice area yielded a statistically significant p-value (.04), and suggests a greater occurrence of this problem in industry as compared to public accounting and government. A review of the underlying data indicate the improper use of statistical evaluation techniques for approximately 25 percent of industry samples, 19 percent of public accounting samples, and 17 percent of government samples. Sampling Literature Sources Consulted Figure 1 reports the sampling literature sources consulted by respondents seeking guidance on sampling and evaluation procedures. All respondents relied on employer standards when seeking guidance and most reported reliance on AICPA literature (Audit Sampling Guide and AU Section 350 of the Audit Standards). Respondents reported much lower levels of reliance on standards issued by the Institute of Internal Auditors (29 percent). However, the usage of this source varied depending on auditor practice area. For example, 100 percent of industry respondents relied on IIA standards, whereas none of the respondents from public accounting or government relied on these standards. As expected, a Chi-square test of independence yielded evidence of statistically significant (p-value = .001) differences in usage rates across audit practice areas. Formal College and Post-College Training Table 2 reports the extent of college and post-college training received in sampling and evaluation procedures. Panel A indicates that almost all respondents reported at least some college-level and post-college-level training in statistical and nonstatistical methods. Because a number of studies report biases in nonstatistical selections and nonstatistical evaluations (see for example Hall et al. 2001; Ponemon and Wendell 1995; Kinney and Uecker 1982), we asked respondents about the extent of training in bias avoidance. Panel B of Table 2 indicates that 8 percent of respondents received formal training that identified and suggested procedures for avoiding selection biases when
using nonstatistical selection procedures. Also, 7 percent of respondents received formal training that identified and suggested procedures for avoiding nonstatistical sampling evaluation biases. Chi-square tests of independence found no evidence that the extent of training varied by auditor practice area (public accounting, industry, or government). Nonstatistical Selection Biases and Mitigating Procedures When asked about the potential for selection biases in nonstatistical selections, respondents reported that such biases, if any, are perceived to be minor. Using a seven-point scale (1 = no effect, 4 = some effect, 7 = great effect) respondents indicated the degree to which the color, size, and physical location of population items influence the composition of nonstatistical samples. Published research suggests these physical characteristics do significantly influence sample selections (see Hall et al. 2001; Hall et al. 2000; Yates 1937) and yet respondents provided average ratings of 1.3 for color, 1.4 for size, and 1.6 for convenience of location. These low response values indicate a belief that these factors do not significantly influence nonstatistical sample selections. Oneway ANOVAs for color, size, and location found no statistically significant evidence that perceptions differed by auditor practice area (p-values = .3200, .9804, and .0817, respectively). None of the respondents reported using any procedures to mitigate selection biases when using nonstatistical selection methods.
Findings This study’s principal contribution is its description of extant audit sampling practice. These research findings help to document the state of generally accepted sampling practice for auditors in public accounting, industry, and government. Except for differences in the overuse of statistical evaluations and the degree of reliance on IIA standards, auditors in all three practice areas report similar procedures.6 Reliance on nonstatistical methods and AICPA literature is common in all areas of audit practice. With respect to sample evaluations, data collected in this study suggest that some audit applications improperly rely on formal statistical methods to evaluate nonstatistical samples. Because formal statistical evaluation methods make important assumptions about selection probabilities for population elements, reliance on these methods is appropriate only when using selection techniques that provide the appropriate selection probabilities (e.g., statistical selection techniques). Using a nonstatistical selection method that does not provide the required selection probabilities could materially misstate the sampling risk estimates provided by a formal statistical evaluation method and lead to auditor decision error. Almost all respondents reported some training in statistical and nonstatistical methods, but relatively few reported receiving substantial training in these areas. Very few respondents reported receiving training in the avoidance of biases when nonstatistical selections and evaluations are used. None reported using debiasing procedures such as increased sample size and stratification when employing nonstatistical selection methods. This failure to use mitigating procedures likely results from two factors: (1) auditors apparently do not believe that nonstatistical selections are materially biased, and (2) auditors do not receive meaningful instruction in debiasing methods. Despite auditor perceptions, several well-designed studies document biases in nonstatistical sample selections and evaluations (see Hall et al. 2000; Ponemon and Wendell 1995; Butler 1986). Although courts accept evidence from nonstatistical applications because of their general acceptance in the profession, these same courts may ask about the procedures used to minimize potential biases (Federal Judicial Center 1994, 240). The absence of meaningful debiasing procedures for nonstatistical sample selections and evaluations may create a legal exposure. The finding that auditors in industry and government place substantial reliance on AICPA sampling literature is understandable given the lack of published literature from other areas of the accounting profession. However, AICPA sampling standards focus on the needs of auditors in public accounting performing financial statement audits. Reliance on these standards by auditors in industry and government may produce undesirable outcomes since the objectives and circumstances of sampling applications in industry and government often differ from those in public accounting.
Implications for Practice and Education Auditors who rely on nonstatistical methods can safely assume that their sampling practices place them in the mainstream. However, these individuals should consider the need for debiasing techniques and restrict the use of statistical evaluations to applications that rely on statistical (random) selection methods. Given the heavy use of nonstatistical methods in practice, university educators should devote additional class time to nonstatistical methods, including information on the weaknesses of these techniques and methods for dealing with these weaknesses. In addition, employers should provide more extensive instruction in the use of nonstatistical methods. Standard-setting bodies for auditors in industry and government should develop detailed sampling standards tailored to the needs of auditors in these practice areas. Also, given the predominance of nonstatistical methods, standard-setting bodies in public accounting, industry, and government should include in their sampling standards a more detailed treatment of nonstatistical methods, their potential weaknesses, and techniques for overcoming these weaknesses. At present, the audit research literature documents several important sampling problems including judgmental errors in evaluating sample results, the practice of error isolation, the failure of auditors to adequately consider sampling risk and sample size in their sample evaluations, and the presence of selection bias in nonstatistical sample selections (Hall et al. 2001; Elder and Allen 1998; Hitzig 1995). Revisions to sampling standards should provide guidance on dealing with these sampling problems. Suggested Resources for Further Reading Table 3 lists suggested readings for practitioners and students seeking more information about sampling. Panel A lists books and monographs that provide detailed guidance on various aspects of audit sampling. Although most of these sources focus on statistical sampling, Guy et al. (1998) and AICPA (2001) include detailed discussions of nonstatistical sampling procedures. However, these sources do not discuss the potential biases that can occur when nonstatistical sampling is used. Panel B of Table 3 lists selected articles reporting research findings about auditor performance in sampling and evaluation tasks. The article by Smith and Kida (1991) provides a general review of research findings on the heuristics and biases exhibited by auditors. In the context of sampling, these behaviors include anchoring and adjustment, representativeness, neglect of base rates, insensitivity to sample size, information-search biases, and biased recall. The other research articles listed in Panel B focus on specific sampling issues such as sample evaluation errors, error isolation and containment, and biases in sample selections. Future Research Future research in audit sampling might focus on identifying the factors that drive the choice of sampling and evaluation techniques. It may be that efficiency considerations, perceptions of audit risk, and auditor familiarity with alternative techniques determine these choices. We also recommend further investigation of the extent to which auditors utilize statistical evaluation techniques to evaluate nonstatistical samples and the underlying rationale for this practice. Although this study made detailed inquiries about the use of alternative statistical selection methods, we limited our inquiries about evaluation methods to distinguishing between nonstatistical or statistical evaluation. Classical statistical evaluation techniquesinclude cluster, difference, dollar-unit, mean-per-unit, ratio, regression, and stratified estimation. More recently developed evaluation techniques include various approaches to resampling, such as bootstrap, jackknife, and permutation. Future research should address the frequency of use of these procedures and the circumstances that prompt their use. To assess the degree of congruence between audit education and practice one could also survey audit educators about the coverage of audit sampling in their college-level courses. Issues addressed might include the specific topics covered and the reasons for covering them. Given the widespread use of nonstatistical techniques, further research on these methods seems appropriate. Issues to be investigated include the determination of sample sizes, the factors that influence sample selections, how auditors consider sampling risk and tolerable error in evaluating audit sample results, and criteria for identifying effective debiasing techniques.
