AUDIT
ANALYTICS
aicpa.org | cpa.com
AICPA Assurance Services Executive Committee
The mission of the AICPA Assurance Services Executive Committee (ASEC)
is to assure the quality, relevance, and usefulness of information or its
context for decision-makers and other users by (1) identifying and prioritizing
emerging trends and market needs for assurance, and (2) developing related
assurance methodology guidance and tools as needed. ASEC achieves its
mission by:
providing guidance and leadership in identifying and prioritizing
signicant emerging assurance trends and market needs while engaging
users, preparers, and inuencers toward action;
developing assurance guidance by creating suitable criteria when
necessary, and/or performance guidance, as appropriate;
communicating new assurance methodologies, guidance, and
opportunities to our members and the profession on a global basis; and
creating alliances with industry, government, or other specialized groups
to improve CPA access to new assurance opportunities.
For additional information on the AICPA’s Assurance Services Executive
Committee please visit aicpa.org/ASEC.
AICPA Business Reporting, Assurance and Advisory Services Team
The overarching role of the AICPA’s Business Reporting and Assurance &
Advisory Services Team is to provide leadership oversight, direction and
visioning for emerging business reporting and assurance issues and initiatives
that are identied and addressed through input from AICPA members,
committees and staff.
For more information on the Business Reporting, Assurance and Advisory
Services Team initiatives, please visit aicpa.org/AAServices.
CONTINUOUS
AUDIT
AUDIT
ANALYTICS
AUDIT ANALYTICS and CONTINUOUS AUDIT
and
AUDIT
ANALYTICS
AUDIT
CONTIN UOUS
Looking Toward
the Future
17970-344_Audit Analytics_final.indd All Pages 7/9/15 10:14 AM
17970-344
AUDIT
ANALYTICS
CONTINUOUS
AUDIT
and
Looking Toward
the Future
17970-344_Audit Analytics_TitlePage.indd 1 7/9/15 10:16 AM
Notice to Readers
Audit Analytics and Continuous Audit: Looking Toward the Future does not
represent an ofcial position of the American Institute of Certied Public
Accountants, and it is distributed with the understanding that the author
and publisher are not rendering legal, accounting, or other professional
services in the publication. This book is intended to be an overview of the
topics discussed within, and the author has made every attempt to verify
the completeness and accuracy of the information herein. However,
neither the author nor publisher can guarantee the applicability of the
information found herein. If legal advice or other expert assistance is
required, the services of a competent professional should be sought.
Copyright © 2015 by
American Institute of Certied Public Accountants, Inc.
New York, NY 10036-8775
All rights reserved. For information about the procedure for requesting
permission to make copies of any part of this work, please email
written and mailed to the Permissions Department, AICPA, 220 Leigh
Farm Road, Durham, NC 27707-8110.
1234567890SP198765
ISBN: 978-1-94354-608-4
NOTICE TO READERS
This publication has not been approved, disapproved, or otherwise acted
upon by any senior technical committees of, and does not represent an
ofcial position of, the American Institute of Certied Public
Accountants. It is distributed with the understanding that the
contributing authors and editors, and the publisher, are not rendering
legal, accounting, or other professional services in this publication. If
legal advice or other expert assistance is required, the services of a
competent professional should be sought.
iii
TABLE OF CONTENTS
Page
Preface xi
Acknowledgements xiii
Author Biographies xv
Part I Essays
1 Continuous Auditing—A New View 3
Nancy Bumgarner, Miklos A. Vasarhelyi
1. Introduction—Continuous Assurance the Theory 3
1.1 Continuous Process Auditing 4
1.2 Conceptualizing Various Elements of CA 6
1.3 Guidance on Continuous Auditing 13
2. The Elements of Continuous Assurance Revisited 13
2.1 Continuous Auditing Versus Continuous
Monitoring 13
2.2 The Elements of Continuous Audit 17
3. Information Technology and the Auditor 19
3.1 Evolving Database Audit Conceptualization 22
3.2 Incremental Technological Change 23
3.3 The Audit Data Standard 24
4. The New Continuous Audit 26
4.1 Assurance Level 28
4.2 Time Focus 29
4.3 Time Interval 30
4.4 Data Source 31
4.5 Chosen Procedure 32
4.6 Choice of Assertion 33
4.7 Analytic Method 33
4.8 Assurance Entity 35
5. Questions Regarding Some Auditing Concepts
in the Modern Environment 35
5.1 Stochastic Opinion Rendering in a World
of Statistics 36
5.2 New Audit Products 37
5.3 Management, Control, Assurance, and
Other Meta-Processes Confusion of Concepts 38
5.4 Independence 39
v
CONTENTS
Page
1 Continuous Auditing—A New View—continued
5.5 Migration of Functions to Automation 39
5.6 The Audit Ecosystem 42
6. Conclusions 46
6.1 The New CA 47
References 49
2 The Current State of Continuous Auditing and
Continuous Monitoring 53
Paul Eric Byrnes, Brad Ames, Miklos Vasarhelyi
Introduction 53
Current Environment 54
Products and Services 55
Promotion Efforts 56
Skills Required 57
Supplemental Findings 58
Conclusions 59
References 60
Appendix—Continuous Auditing and
Continuous Monitoring in Action 60
Introduction 60
SAP Key Performance Indicator 61
DSAS/Audit Command Language 61
DSAS Database 61
Dashboard Feature 63
3 Evolution of Auditing: From the Traditional Approach
to the Future Audit 71
Paul Eric Byrnes, Abdullah Al-Awadhi, Benita Gullvist, Helen
Brown-Liburd, Ryan Teeter, J. Donald Warren, Miklos Vasarhelyi
Introduction 71
A Brief History of Auditing in the United States 72
The Traditional Audit 76
Automating the Audit 77
The Future Audit 78
Embedded Audit Modules 79
Monitoring and Control Layer 80
Audit Data Warehouse 81
Audit Applications Approach 81
vi
CONTENTS
Page
3 Evolution of Auditing: From the Traditional Approach
to the Future Audit—continued
Other Future Audit Considerations 82
Conclusion 83
References 84
4 Reimagining Auditing in a Wired World 87
Paul Eric Byrnes, Tom Criste, Trevor Stewart, Miklos Vasarhelyi
Overview 87
Introduction: Blue Sky Scenario 88
Using Technology to Transform Auditing 91
Technology Enablers 91
Audit Opportunities 92
More Effective Audit Data Analytics 92
More Assurance 95
Auditing With Big Data 96
Continuous Auditing, Continuous Assurance 97
More Effective Fraud Detection 98
Reducing False Positives 98
Audit Process Re-Engineering: An Example 99
Making It Happen 100
Encouraging Audit Research and Development 100
Providing Guidance and Updating Auditing
Standards 101
Encouraging and Recognizing New Resource Models 101
Blue Sky Scenario Revisited 102
References 102
5 Data Analytics for Financial Statement Audits 105
Trevor R. Stewart
Abstract 105
The Audit Context 105
DA and Generally Accepted Auditing Standards 106
Audit Applications of DA 108
Understanding the Entity, and Risk Assessment 108
Performing Substantive Analytical Procedures 109
Analyzing and Testing Populations of
Detailed Transactions and Balances 110
Considering and Testing for Fraud 111
vii
CONTENTS
Page
5 Data Analytics for Financial Statement Audits—continued
Testing the Operating Effectiveness of
Internal Control 112
Inquiry 112
A Look Ahead: Cognitive Computing in the Age
of Big Data 112
Utilizing Big Data 112
Cognitive Computing 113
Upping Our Game 114
Illustrative Examples 115
Example 1: Simple DA Visualization 115
Example 2: Financial Ratio Peer Analysis 120
Multivariate Ratio Analysis 125
References 128
6 Managing Risk and the Audit Process in a World of
Instantaneous Change 129
Paul Byrnes, Gerard Brennan, Miklos Vasarhelyi, Daehyun Moon,
Satyajeet Ghosh
Abstract 129
Introduction 130
CRMA Architecture—Overview 130
CRMA—General Process 132
CRMA—More Detailed Considerations 133
Risk Identication and Analysis 133
KRI Development and Implementation 134
Auditor Response to Changing Risk Levels 136
Management Response to Changing Risk Levels 137
Hypothetical Illustration of CRMA in Use 138
Systematic Implementation of Risk
Management and Assessment in a Process 139
Conclusion 142
References 142
Part II Case Studies
A Developing Continuous Assurance at Siemens 147
Ann F. Medinets, Jason A. Gross, Gerard (Rod) Brennan
Traditional Internal Audit 148
Continuous Controls Monitoring 148
viii
CONTENTS
Page
B Implementing Continuous Auditing and Continuous
Monitoring in Metcash—Change, Capabilities, and Culture 157
Glen Laslett, Catherine Hardy
Introduction 157
Value Proposition: Identifying the Need and
Addressing the Business Challenge 158
The Importance of Architecture 159
Denitions and Applications of CA/CM in Metcash 161
An Example Application: The Leave
Continuous Monitoring Routine (CMR) 162
Moving Forward—Key Risk Indicators 163
Challenges and Lessons Learned 164
Conclusion 166
References 166
C Increasing Audit Efciency Through Continuous
Branch KPI Monitoring 169
Carlos Elder de Aquino, Eduardo Miyaki, Nilton Sigolo, Miklos A.
Vasarhelyi, Paul E. Byrnes
Abstract 169
Introduction 170
The Process at SAB 170
Potential Enhancements 171
Conclusions 172
References 172
D Implementing Continuous Monitoring at Vodafone Iceland 175
Mar
´
ıa Arth
´
ursd
´
ottir, H
¨
orður M
´
ar J
´
onsson, Sindri Sigurj
´
onsson
Introduction 175
Continuous Monitoring in Vodafone Iceland 176
Revenue Leakage 177
Process of Monthly Financial Closing 178
The Billing Process 179
Fraud Monitoring 181
Customer Relationship Management 181
Culture Change and Enhanced Quality of
Work Flow 182
Challenges and Learning 183
The Future 183
Conclusion 184
ix
PREFACE
The world is evolving and so is the Accounting Profession (the
profession). Recent technological advances offer both challenges and
opportunities that will change the way CPAs operate into the foreseeable
future. In order to stay on top of these new and emerging trends, we need
to align the profession to continue to meet client needs and expectations.
As a step in this direction, the AICPA Assurance Services Executive
Committee (ASEC) Emerging Assurance Technologies Task Force
Continuous Assurance Working Group has developed this book, Audit
Analytics and Continuous Audit: Looking Toward the Future, which focuses
on continuous auditing, continuous control monitoring, and advanced
analytics.
In 1999, the Canadian Institute of Chartered Accountants (CICA) and the
AICPA developed a research report entitled Continuous Auditing.This
report discussed the viability of continuous audits, described a
conceptual framework for conducting them and identied signicant
issues that auditors would likely encounter when performing this type of
work. Audit Analytics and Continuous Audit: Looking Toward the Future,is
intended to be an update to the CICA and AICPA research report,
Continuous Auditing.
Audit Analytics and Continuous Audit: Looking Toward the Future is a
compendium of essays written by different subject matter experts that
expands upon the CICA and AICPA research report to discuss the
following:
r
The theory of modern continuous assurance
r
The current state of continuous auditing and continuous
monitoring
r
The evolution of auditing and what the future could look like
r
Audit analytics
r
Continuous risk monitoring techniques
The book also includes detailed examples and case studies of companies
today that have implemented elements of continuous auditing and
continuous control monitoring into their day-to-day operations.
xi
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Each author brings unique perspectives and insights to each of the essays
included within this book. The authors are made up of individuals in
public accounting, business and industry, as well as academia. Each
author shares a clear vision of the future, and is dedicated to the
advancement of the profession.
xii
ACKNOWLEDGEMENTS
AICPA Assurance Services Executive Committee
Robert Dohrer, Chair
Dorsey Baskin
Bradley Beasley
Greg Bedard
Nancy Bumgarner
Chris Halterman
Charles E. Harris
Don Kluthe
Chris Kradjan
Michael Ptasienski
Beth A. Schneider
Miklos Vasarhelyi
Deetra B. Watson
Don Pallais (Observer)
AICPA Staff
Amy Pawlicki
Director, Business Reporting, Assurance & Advisory Services
Dorothy McQuilken
Manager, Business Reporting, Assurance & Advisory Services
Rachelle Drummond, CPA
Technical Manager, AICPA Peer Review Program
Tanya Hale, CPA
Technical Manager, Business Reporting, Assurance & Advisory Services
xiii
AUTHOR BIOGRAPHIES
Abdullah Alawadhi, PhD
Assistant Professor, Kuwait University
Abdullah Alawadhi is an assistant professor in the College of Business
Administration (CBA) at Kuwait University. He received his PhD from
Rutgers University in 2015. His research interests include data
visualization, audit analytics and big data. For four years he worked at
the Continuous Auditing and Reporting Laboratory (CAR Lab) at
Rutgers University, conducting many research oriented projects.
Abdullah obtained his bachelor degree, majoring in Accounting, from
Kuwait University in 2008. In 2011, he graduated from the University of
Pittsburgh, Katz Business School, and obtained his Master of Science
degree in Accounting. Abdullah presented in several conferences,
including the 28th World Continuous Auditing and Reporting
Symposium (28WCARS) and the American Accounting Association
(AAA) 2014 Mid-Atlantic Region Meeting. In addition, he has several
working papers on the area of audit analytics and data visualization.
Brad Ames, CPA, CISA, CRMA
Director, Internal Audit, Hewlett-Packard
Company
Brad Ames’s role involves close collaboration with Hewlett-Packard’s
governance groups, compliance functions, customers and external
auditors in order to gain an ongoing view of emerging risk
enterprise-wide. His team is responsible for innovating and deploying
continuous auditing solutions for measuring risk to the business and
shortening the time to management action.
Brad is an active CPA, certied information systems auditor (CISA), and
holds a Certication in Risk Management Assurance
®
(CRMA
®
). He has
a bachelor of science from LeTourneau University and is a member of
the AICPA and Institute of Internal Auditor’s Professional Issues
Committee.
xv
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Carlos Elder Maciel de Aquino
Statutory Director, Diagnotics of America S/A
Carlos Elder Maciel de Aquino is currently statutory director at
Diagnostics of America S/A and is currently responsible for the areas of
accounting, receivables, accounts payable and corporate registration. He
was a partner at KPMG and previously worked at institutions such as
Ita
´
u Unibanco and Unibanco, where he held positions such as director of
internal auditing, executive director of leasing and managing director of
complementary social securitie. He was also the sector director of
Febraban’s internal auditing department, was a member of Redecard’s,
Tecban’s and Interbanco’s (Paraguay) auditing committees. He earned a
degree in accounting from UFPE, a graduate degree in Financial
Administration from FESP-PE and in Economic Engineering from
Universidade Cat
´
olica de Pernanbuco (UNICAP-PE). He has an
Executive MBA in Financing from Instituto Brasileiro de Mercado de
Capitais (IBMEC) and a Controller MBA from Universidade de S
˜
ao Paulo
(USP) and is an auditing course professor for the MBA in Internal
Auditing at Fundac¸
˜
ao Instituto de Pesquisas Cont
´
abeis, Atuariais e
Financeiras (FIPECAFI).
Mar
´
ıa Arth
´
ursd
´
ottir
Head of Financial Planning and Analysis,
Vodafone Iceland
Mar
´
ıa Arth
´
ursd
´
ottir is currently a part-time student at Reykjavik
University in Management Accounting and Business Intelligence (MABI).
She graduated from University of Iceland in Finance, Cand.Oecon, 1996.
Mar
´
ıa has been employed at Vodafone Iceland since 2006 and has
extensive experience in management accounting, business intelligence,
revenue assurance, continuous monitoring, planning and analysis. She
recently introduced and implemented rolling forecasting and Beyond
Budgeting in Vodafone Iceland.
Gerard (Rod) Brennan, CFE, PhD
NA Risk & Internal Control Ofcer, Siemens Corp.
Rod Brennan is a practitioner, frequent speaker, and published researcher
on the topic of continuous auditing and monitoring. He is currently NA
risk & internal control ofcer for Siemens and an adjunct professor in the
Rutgers University MBA program teaching Advanced Auditing and Info
Technology.
xvi
AUTHOR BIOGRAPHIES
He is a passionate advocate of using technology to audit and monitor and
is working with researchers from around the world to develop a
continuous auditing and monitoring culture and technology for Siemens.
Rod successfully defended his PhD thesis The Use of Intelligent Software to
Enable Continuous Auditing. The research work included the design and
development of a proof of concept ERP continuous auditing software
model (using SAP) incorporating some of the latest continuous auditing
research concepts. The model was co-developed with Rutgers
University’s Continuous Auditing Research Laboratory (CAR Lab)—a
leading continuous auditing research group.
Rod has been actively involved in the design and implementation of
automated auditing and monitoring solutions using a variety of software
applications and worked on a centralized risk and internal control
solution for Siemens. Siemens operates in diverse business sectors
throughout the world in more than 175 countries.
Helen L. Brown-Liburd, PhD, CPA
Assistant Professor, Rutgers University
Helen L. Brown-Liburd received her PhD from the University of
Wisconsin-Madison and a bachelor of business administration in
accounting from Baruch College. Her research focuses on issues and
factors that inuence auditors’ judgment and decision making related to
nancial reporting. She has published in Auditing: A Journal of Practice and
Theory, Accounting Horizons, Journal of Business Ethics,andIssues in
Accounting Education. Her teaching experience includes auditing, AIS,
and nancial accounting.
Helen has more than 16 years of experience in such diverse areas as
auditing, nancial and operating reporting, and analysis and project
management. She has worked for Bristol-Myers Squibb (BMS) as a
manager on several company-wide teams established to evaluate and
redesign major company-wide processes and she also served as an
internal audit manager responsible for supervising and monitoring
worldwide audits. She also worked for Pepsi Cola Company as manager
of special projects where she researched, developed, and implemented
accounting policies and procedures and performed nancial reporting for
acquisitions. She began her career in public accounting as a staff auditor
for Main Hurdman (now KPMG) and later moved to Arthur Young (now
EY) where she was promoted to audit manager.
Nancy Bumgarner, CPA
Partner, KPMG
Nancy Bumgarner is an audit partner with KPMG and has served in a
number of national, international, and global roles over her career. She is
xvii
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
currently in KPMG’s Department of Professional Practice in the audit
technical group and has worked in the Houston, Sydney, Tulsa, and
Oklahoma City ofces and completed a rotation to KPMG’s Global
Services Centre where she developed new and innovative technologies
and audit methodology that facilitate effective and efcient audits in
KPMG member rms around the world. While in the global role, Nancy
created a new electronic audit tool that harnessed data and analytics with
advanced modeling and algorithms that rene risk assessment and audit
approaches.
While in Australia, Nancy took a leading role assisting international
teams and clients with IFRS and U.S. GAAP differences, U.S. auditing
standards, nancial reporting, and SEC matters. Nancy is a frequent
speaker and panelist at various technical, diversity, and rm-sponsored
meetings. Nancy is a member of KPMG’s Women’s Advisory Board and
KPMG’s Partner Insight Committee on People.
Nancy is a member of the AICPA Assurance Services Executive
Committee and serves on its Continuous Assurance Working Group.
Paul Byrnes, CMA
PhD Student, Rutgers University
Paul Byrnes has an advanced degree in accounting and four-year degrees
in accounting, management, and psychology. In addition, he is a
Certied Management Accountant with about 15 years of relevant work
experience in both the accounting and management elds.
Thomas R. Criste, CPA
Retired Partner, Deloitte & Touche LLP
Tom Criste recently retired as a partner with Deloitte & Touche LLP, with
more than 30 years of experience in nancial and IT audit, risk
management, and internal controls. He served many of the rm’s largest
multinational clients in a number of industries, including automotive,
aerospace, consumer goods, and technology. He also served as the rst
chief learning ofcer for Deloitte & Touche LLP, the audit and assurance
rm of Deloitte, where he led a successful effort to establish Deloitte
University, one of the largest privately-owned learning centers in North
America.
Tom is currently a member of the accounting faculty at the Ross School of
Business of the University of Michigan in Ann Arbor, MI, with a focus on
accounting information systems and auditing. He is also on the Advisory
Board of Rutgers’ Continuous Auditing and Reporting Laboratory (CAR
Lab). He currently serves on an audit data analytics task force of the
xviii
AUTHOR BIOGRAPHIES
AICPA Assurance Services Executive Committee (ASEC) as well as the
continuous assurance working group.
Satyajeet Ghosh, MS, MSE, MBA, CIA, CISA, CFE
SVP, General Auditor and Risk Management, CA
Technologies, NY
Satyajeet (Saty) Ghosh joined CA Technologies in January of 2011 as the
general auditor responsible for assurance, process simplication,
Sarbanes-Oxley (SOX) compliance, and enterprise risk management.
Prior to joining CA, Satyajeet held numerous management and executive
roles in risk management, software development, technology capital
management, business transformation, and mergers and acquisitions in
various industry verticals. He has worked at AT&T Bell Labs, Dun &
Bradstreet, IDT Corporation, Telcordia, Fortent and United Engineers. He
is currently on the Advisory Board of CAR Lab and Senior Fellow at
Rutgers Graduate School of Business, and was a lecturer at numerous
universities. His current interests include predictive analytics, controls
monitoring, and risk maturity models.
Satyajeet holds an MBA from Columbia Business School and graduate
degrees in computer science and engineering from Drexel University and
the University of Pennsylvania, respectively, and an under-graduate
degree in engineering from the Indian Institute of Technology, Kanpur
(India). He is also a current and past member of various professional
organizations that include IIA, ISACA, ACFE, and IEEE.
Jason A. Gross, CPA, CIA, CFE, CISA, ACDA
Vice President, Controls Management, Siemens
Financial Services, Inc.
Jason A. Gross is responsible for ensuring an effective and efcient
internal control framework, leveraging his seven years of experience as
Siemens’ vice president, internal audit. In this capacity, Jason and his
department focus on the design, implementation, and monitoring of
controls via the continuous controls monitoring (CCM) program as well
as the management and oversight of various control programs, including
the SOX program. Prior to joining Siemens Financial Services, Inc. in
2002, Jason had ve years of internal audit experience with AT&T
managing internal audit activity of various business units and four years
of public accounting experience at the CPA rm of Weiser LLP.
Jason is also an experienced speaker on SOX and internal controls and
has presented on several occasions for the Equipment Leasing and
Finance Association (ELFA), MIS Training Institute, the Bank
xix
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Administration Institute (BAI), and for the Institute of Internal Auditors
(IIA) at several IIA conferences. These IIA conferences included the
General Audit Management (GAM), Financial Services, the International
IIA conference, as well as the 2008 IIA webcast, Continuous Auditing: What
Works Best. Jason has received the distinction of being named to the list of
All-Star speakers for the IIA 2006 All Star Conference. In addition, he has
gained recognition in the industry as a subject matter expert on the topics
of utilization of audit tools including audit management systems, data
analytic tools and methodology, and continuous monitoring and
auditing. Jason was featured in the Best Practices of Highly Successful
Auditors by ACL and has been named to the 2009 100 Most Inuential
People in Finance of Treasury and Risk.
In addition to speaking on the topics of internal controls, SOX, and audit
tools, Jason has also authored articles in the Internal Auditor publication
of the IIA and in the ELT: The Magazine of Equipment Leasing & Finance
publication of the ELFA.
Benita Gullkvist, DSc
Associate Professor, Hanken School of
Economics, Finland
Benita Gullkvist received her DSc degree from Abo Akademi University
in 2005. Her research interests include behavioral, social, and
organizational issues related to accounting, especially within the eld of
accounting information systems. She has recently published in journals
such as European Accounting Review and Critical Perspectives on Accounting
among others. Funding from the Fulbright Center, Finland made it
possible to take part in auditing research at Rutgers Business School, USA
in 2011–2012.
Catherine Hardy, PhD
Senior Lecturer, University of Sydney
Catherine Hardy is a senior lecturer at the University of Sydney Business
School in the Business Information Systems Discipline. Her research
interests focus mainly on the complex and changing relationships
between technical innovation, organizational change and governance,
and accountability systems. Catherine’s current research project is a
case-based study on the adoption, implementation, and evaluation of
continuous auditing and continuous monitoring in Australian
organizations. She has extensive teaching, curriculum development, and
program management experience in a wide range of information systems
subject areas including information governance, information protection
and assurance, accounting information systems, and project
xx
AUTHOR BIOGRAPHIES
management. Prior to joining academia, Catherine was an accountant in
the nancial services industry.
H
¨
orður M
´
ar J
´
onsson
Partner, Expectus
H
¨
orður M
´
ar J
´
onsson is a management consultant at Expectus with more
than 15 years of experience in information technology focusing on
business intelligence, nancial planning, and continuous monitoring.
H
¨
orður’s unique background, combining IT and management
consulting, enabled him to develop a unique continuous monitoring
solution, exMon. H
¨
orður has a B.Sc. in computer science.
Glen Laslett, CA, CIA
Retired, Metcash Ltd.
Glen Laslett has recently retired as group business assurance manager at
Metcash Ltd.
Glen’s areas of business expertise include risk, internal audit, continuous
assurance, and process management. Glen has a particular interest in
continuous monitoring (CM) and, during his 14 years at Metcash,
developed an extensive CM framework that delivered substantial
benets to the business. The resulting CM framework comprises more
than 100 fully automated routines that deliver evidence of potential
control breakdowns and transactional anomalies to the business and
subsequently monitors their remediation. Prior to joining Metcash, Glen
worked in the nancial services and hospitality industries. Glen is a
chartered accountant and a certied internal auditor.
Ann Medinets, PhD
Professor, Rutgers University
Ann Medinets received her MBA and PhD from Rutgers University. She
teaches courses in managerial accounting, intermediate accounting, cost
accounting, and corporate governance at Rutgers. Her research interests
include resource allocation, management information systems, and
shareholder rights.
Daehyun Moon, CPA
PhD Student, Rutgers University
Daehyun Moon is currently a PhD student in accounting information
systems at Rutgers University and teaching accounting courses at
University of La Verne. He earned a bachelor’s degree in accounting from
xxi
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Indiana University Bloomington and an MBA in nance from Indiana
State University. He holds a CPA certicate in Illinois.
Eduardo Hiroyuki Miyaki, CIA, CCSA, CFSA
Managing Director of Internal Auditing, Ita
´
u
Unibanco Holding
Eduardo Hiroyuki Miyaki is currently the managing director of the
Internal Auditing function at Ita
´
u Unibanco Holding, responsible for
auditing investment banking, treasury, corporate, and small to midsized
enterprise operations and its international unities (retail and corporate).
He coordinates all continuous auditing initiatives at the bank. Prior to his
experience as internal auditor, he coordinated the implementation and
managed the Anti-Money Laundering Program at Banco Ita
´
uS.A.
He holds a degree in civil engineering; a specialization in nancial
management at Fundac¸
˜
ao Getulio Vargas; and an MBA in nance and
international business at New York University, Leonard Stern School of
Business, USA. He also holds CIA, CCSA, and CFSA certications.
Nilton Sigolo
Research Fellow, Rutgers Business School
Nilton Sigolo is a fellow at the Rutgers Business School’s CAR Lab and
worked for 35 years at the internal audit of Unibanco and Itau-Unibanco
in Brazil. During the last 13 years, he headed the continuous auditing
department. He participated in the Brazilian’s Bank Association
(FEBRABAN) New Bank Auditing Concepts task force and book in
collaboration with Deloitte. He has published in the Internal Auditor and
has performed recently a set of independent audits of different nature.
Sindri Sigurj
´
onsson
Partner, Expectus
Sindri Sigurj
´
onsson is a management consultant at Expectus. He has
worked with some of the largest companies in Iceland focusing on
strategy, performance management, nancial planning, business
intelligence, and business process re-engineering through the use of CM.
He has more than 15 years’ experience in assisting companies in
streamlining their processes and improving their performance.
Previously he was the director of business development at Shell Iceland,
where he implemented CM, and executive director of production
development at Actavis. Sindri has a M.Sc. in operational research and
B.Sc in industrial engineering.
xxii
AUTHOR BIOGRAPHIES
Trevor R. Stewart, CA, PhD
Retired partner, Deloitte & Touche LLP; Senior
Research Fellow, Rutgers Business School
Born and educated in South Africa, Trevor Stewart joined Deloitte in
Johannesburg, working there and in London before transferring to New
York in the early 1980s. In New York, he served in various national and
global roles until his retirement in 2009 after 38 years with the rm, 31 as
a partner. He is a chartered accountant (South Africa), has a bachelor of
science (honors) degree in mathematics from the University of Cape
Town and, post retirement, completed a PhD at VU University
Amsterdam with a thesis on audit assurance and component materiality
in group audits—work that was also published in The Accounting Review
in a paper with Professor William R. Kinney, Jr.
Trevor started Deloitte’s international audit technology research and
development center in Princeton, NJ, which he led for over a decade. He
developed, with Kenneth W. Stringer, a technique (STAR) that uses
multiple regression and other statistical methods for performing
analytical procedures, together with related software, and co-authored
Statistical Techniques for Analytical Review in Auditing (Wiley, 1996). He
served on the rm’s global audit technical policies and methodologies
committee until his retirement.
Trevor has served on several AICPA committees and task forces,
including the 2008 Audit Sampling Guide task force for which he wrote
the companion technical notes. He currently serves on an Audit Data
Analytics task force of the Assurance Services Executive Committee
(ASEC). He was vice-president, practice, of the Auditing Section of the
American Accounting Association, 2006–2008. He currently serves on the
advisory board of Rutgers’ CAR Lab.
Ryan Teeter, PhD
Professor, University of Pittsburgh
Ryan Teeter teaches accounting information systems at the University of
Pittsburgh. He received his PhD from Rutgers University in New Jersey
and has conducted audit research with Siemens and Procter & Gamble.
He specializes in remote auditing and audit automation.
Miklos A. Vasarhelyi, PhD
Professor, Rutgers University
Miklos A. Vasarhelyi holds a PhD in MIS from UCLA, an MBA from MIT,
and a BS in economics and electrical engineering from the State
xxiii
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
University of Guanabara and Catholic University of Rio de Janeiro.
Professor Vasarhelyi is currently the KPMG Distinguished Professor of
Accounting Information Systems and director of the Rutgers Accounting
Research Center (RARC) CAR Lab. He has published more than 200
journal articles, 20 books, and directed over 30 PhD theses. He is the
editor of the Articial Intelligence in Accounting and Auditing series and the
Journal of Information Systems. Professor Vasarhelyi has taught executive
programs on electronic commerce to many large international
organizations including GE, J&J, Eli Lilly, Baxter, ADL, Volvo, Siemens,
Chase Bank, and AT&T. Professor Vasarhelyi is credited with the original
continuous audit application and as the leading researcher in this eld.
The CAR Lab’s projects include among others Siemens, KPMG, P&G,
AICPA, CA Technologies, and Itau-Unibanco. He was the co-recipient of
the AAA outstanding educator of 2013 and ISACA’s Wasserman award
of 2012.
J. Donald Warren, Jr., PhD
Assistant Professor, University of Hartford
J. Donald Warren, Jr., PhD, is an assistant professor in the Barney School
of Business at the University of Hartford. He previously taught in the
Rutgers Business School and served as the director of the Masters of
Accountancy in Financial Accounting. Professor Warren retired from
PricewaterhouseCoopers LLP after a career of 31 years. He served in
many capacities with PwC, including being responsible for the direction
of the IT audit practice and serving as a national consulting partner on
accounting and auditing matters and the rm’s liaison to the SEC.
Additionally, in that capacity, one of his responsibilities was to review
and interpret the AICPA Code of Professional Conduct which contains
the ethical standards for CPAs. He co-authored PwC’s SEC Manual and
the third edition of the Handbook of IT Auditing. His other work
experience includes the US General Accountability Ofce and FASB. His
research interests include continuous audit methodologies and processes
and their related technologies.
xxiv
PART I
Essays
1
ESSAY 1
Continuous
Auditing—A New
View
Nancy Bumgarner, CPA
Miklos A. Vasarhelyi, PhD
1
1. INTRODUCTION—CONTINUOUS
ASSURANCE THE THEORY
2
This volume is intended as an update on the report Continuous Audit
(also called Red Book) published by the CICA and AICPA in 1999. In that
volume, some basic principles and a vision were presented that served as
a basis for additional guidance work by the Institute of Internal Auditors
(IIA) in 2005 and the Information Systems Audit and Control Association
(ISACA) in 2010. Fifteen years after that 1999 report, this volume presents
a much different state-of-the-art, and this essay proposes an expanded set
of concepts largely adding to Vasarhelyi and Halper (1991) and joining it
with an increasing set of experiences and literature from practice and
academia. The evolution of IT, the emergence of big data, and the
increasing use of analytics have rapidly changed the landscape and
prole of continuous assurance and auditing.
3
Many of the current audit
1
The suggestions and contributions of professors Michael Alles and Mr. Shrikant Despante are
gratefully acknowledged. This essay also substantively beneted from the suggestions of Messrs.
Bob Dohrer, Chris Kradjan, Dorothy McQuilken, and Beth Schneider.
2
The authors are appreciative for advice and guidance from Professor Michael Alles, the com-
ments of Mr. Shrikant Deshpande, and the research assistance of Ms. Qiao Li.
3
In general the eld of assurance incorporates both the traditional audit as well other types of as-
surance such as SysTrust, WebTrust or assurance on cybersecurity. In this essay continuous assurance
is also taken as potentially a larger set of topics than providing traditional auditing services but on a
more frequent basis. On the other hand, the terms continuous audit and continuous auditing are used
interchangeably.
3
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
standards were initially instituted by legislation based on the Securities
Act of 1933 and the Securities Exchange Act of 1934 and progressively
developed into the current, ever-evolving set of generally accepted
auditing standards, or GAAS. This formalization of "generally accepted"
has had an enormous effect on business practices and consequently large
effects on the social ecosystem.
Within this context, in addition to the external verication of nancial
statements, many contexts in need of third-party verication have risen.
Consequently, organizations developed internal audit departments,
consulting rms introduced auditing services, and some of these needs
are being satised on an ad hoc basis mainly by external audit rms.
Vasarhelyi and Alles (2006), in a study for the AICPA’s Enhanced
Business Reporting (EBR) project, characterized the umbrella of
verication services as "assurance," under which falls a set of services
such as the "traditional (external) audit," internal audit, and much of
what we later in this paper call "audit-like services." Several data analytic
and monitoring functions of the expanded set of activities that we hereby
call continuous assurance have dual or multiple functions serving
assurance, management, and other parties. Guidance on materiality,
independence, and required procedures will eventually be needed to
adapt to the new tools as the environment evolves. This essay illustrates
some of these needs.
Groomer and Murthy (1989) and Vasarhelyi and Halper (1991) have
respectively argued for and demonstrated the desirability and possibility
of "closer to the event" assurance processes. This approach, reecting the
evolution of technology to online, real-time systems, has had slow but
progressive adoption both in practice (Vasarhelyi et al, 2012; ACL 2006;
PWC 2006)
4
and in professional guidance (CICA/AICPA, 1999; IIA, 2005;
ISACA, 2010).
1.1 Continuous Process Auditing
Motivating the need for continuous assurance, Vasarhelyi and Halper
(1991) state: "There are some key problems in auditing large database
systems that traditional auditing (level 1) cannot solve. For example,
given that traditional audits are performed only once a year, audit data
may be gathered long after economic events are recorded." To deal with
these problems, the AICPA/CICA’s Red Book (1999) introduced the
current denition of continuous auditing:
A continuous audit is a methodology that enables independent
auditors to provide written assurance on a subject matter, for
which an entity’s management is responsible, using a series of
4
PricewaterhouseCoopers, Internal Audit Survey; Continuous Audit Gains Momentum, 2006.
4
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
auditor’s reports issued virtually simultaneously with, or a short
period of time after, the occurrence of events underlying the
subject matter. (CICA/AICPA, 1999)
Research studies have provided a much broader perspective on how
technology is changing auditing. Alles, Kogan, and Vasarhelyi (2002)
questioned whether there was an economic demand for continuously
provided assurance and suggested that the more likely outcome is audit
on demand. Alles, Brennan, Kogan, and Vasarhelyi (2006) expanded the
scope of the continuous audit by dividing it into continuous control
monitoring (CCM) and continuous data assurance (CDA). It has also
been shown that many internal audit procedures can be automated, thus
saving costs, allowing for more frequent audits and freeing up the audit
staff for tasks that require human judgment (Vasarhelyi, 1983, Vasarhelyi,
1985; Alles, Kogan, and Vasarhelyi, 2002).
In the last decade of the 20th century, many large companies, prompted
in part by the Y2K concern, replaced their legacy IT systems with new
enterprise resource planning (ERP) systems. These ERP systems are
controlled by extensive control settings while data is organized into
relational databases that are composed of complex, multi-dimensional
tables that are "related" to each other for the creation of reports by
common elds. Users, for highly justiable business reasons, are allowed
to override control settings. Consequently, new assurance needs have
emerged due to the ever increasing difculty of direct observation of
(1) control structures, (2) control compliance, and (3) data.
Control Structure
The ubiquitous usage of ERPs diminished concerns with the adequacy of
control structures as the systems are typically based on best of class
implementation and widely used even though each company will
determine how the ERP control structure will be adopted for
company-specic circumstances. Many questions remain, as the actual
control structure does not only involve the ERP systems but also the
entire manual and IT set of processes (that include many elements aside
from the ERP systems) and their integration. Controls can be overridden
or bypassed by the users, or may not exist at the upstream of the process,
and transactions will be received as legitimate.
Control Compliance
Control compliance, on the other hand, became a much larger problem as
established exible and widely applicable control structures often entail a
very large number of controls and for operational reasons these controls
may have to be temporarily re-parameterized. For example, a particular
checking account may be allowed to go over its credit limit for
5
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
operational reasons. The need to monitor and assure control settings and
the nature of overrides generated a new type of audit objective and
process.
Data
Data is in general stored in ERPs, in les for legacy systems, or in more
recent times in large repositories external to the organization that are
called big data (Vasarhelyi, Kogan, and Tuttle; 2015). The access to these
data for observation, monitoring, or mass retrieval requires the auditor’s
knowledge and extensive use of software tools. This access is not only
technically challenging but also organizationally difcult (Vasarhelyi,
Romero, Kuenkaikaew, and Littley; 2012).
1.2 Conceptualizing Various Elements of CA
Table 1-1 illustrates the uses, purposes, and approach of the expanded
model of continuous assurance differentiating between internal and
external usage and further differentiating between diagnostic, predictive,
and historic usage.
Table 1-1: Users, Purpose, and Approach of the Elements of Continuous
Assurance
Data
assurance
Controls Compliance
Risk
monitoring
and
assessment
Operations
(monitoring)
Who uses
•
Management
X X X X X
•
Audit (internal
or external)
X X X
•
Investors
X
•
Regulators
X X X
Purpose
•
Diagnostic
X X X X
•
Predictive
X X
•
Historic
X X X X X
Primarily performed by
•
Automation
X X X X X
•
Manual
X X X
Each of these elements is discussed in the following sections.
Continuous assurance (CA) has the potential to benet a wide variety of
users. Management will be interested in all aspects, from data assurance
6
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
to monitoring operations. Investors may primarily be interested in data
assurance though, depending on the industry, compliance and risk
monitoring may be equally as important.
CA is well suited for historic analyses, particularly given the speed with
which CA provides information on attributes such as accuracy. Auditors
that provide assurance on historic information will likely be primarily
interested in the ability of CA to be used for such purpose. Access to
sophisticated ERPs and complex data sets create an opportunity for CA
to be used for diagnostic purposes. Where an error or anomaly has been
identied, CA may perform a retrospective diagnostic of the
situation—providing insight and analyses to management.
Diagnostically, CA could also be tied to effectively assessing operational
and structural strengths and weaknesses of an organization—enabling
strategic decisions to be made in a timely manner and with sufcient
context.
Automation is an essential element to CA, though manual involvement
remains important particularly in situations where extensive judgment is
required and where anomalies, exceptions, and outliers are identied.
Continuous Data Audit CDA
Vasarhelyi and Halper called the process of monitoring and constantly
assuring AT&T’s RCAM system continuous audit. The architecture of the
system described in gure 1-1 shows data being (1) extracted from
pre-existing reports, (2) sent to the business units through the remote job
entry network, (3) transferred to an email system, and (4) extracted
through individual text mining programs. This technique, analogous to
what is called today "screen scrapping," was chosen to avoid interference
in the long and complex system process development protocol. All
information was collected from existing reports and placed in a relational
database. This database drove hypertext graphs that were given to
auditors to interact with the system. The several layers of the RCAM
system were represented as owcharts respecting the internal auditors’
documentation practices and experience in data analysis. Many of the
analytics impounded into the system were drawn from knowledge
engineering (Halper, Snively, and Vasarhelyi, 1989) internal auditors and
capturing the calculations they made with paper reports. The
formalization of these processes allowed for their repetition at repeated
frequency, and often reliance on these tests up to the moment that alerts
were generated. Although internal auditors started relying on these
exception reports, they also requested that the source reports be retained
mainly for their traditional audit reports.
7
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 1-1: CPAS as Continuous Data Audit (from Vasarhelyi & Halper,
1991)
Although the idea of a continuous audit was conceptualized initially as a
data monitoring and exception system (Vasarhelyi, 1996), its concept was
expanded in an implementation at Siemens (Alles et al, 2006) as a
reaction to Sarbanes Oxley and the need to issue opinions on the
adequacy of internal controls. This expansion was entitled continuous
control monitoring (CCM).
Continuous Control Monitoring (CCM)
Siemens had over 150 instances of SAP that were reviewed by technical
experts using that narrow guidance of a standardized set of audit action
sheets. These were a formalization of the audit plan to review controls
and features of a particular SAP implementation and were adapted to
each audit instance. Alles et al. (2006) developed a proof of concept tool
where a baseline of control settings would be compared with the actual
congurable control setting every night and auditors would be alerted of
variations. Teeter (2014) extended the original work examining the
potential for automation of not only the deterministic settings of SAP but
a wider set of controls and parameters in the SAP system.
The...essay...investigates the implementation of a comprehensive
continuous controls monitoring (CCM) platform for evaluating
8
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
internal controls within a highly formalized and well-controlled
enterprise resource planning environment. Utilizing the IT audit
plan as a template, auditor expertise as a guide, and manual audit
output as a validation tool, this eld study examines the process of
audit formalization and implementation of CCM at a software
division of a large, multinational corporation. (Teeter, 2014)
The results of the applied effort
5
indicated that 62 percent of the controls
arguably could be formalized, creating the possibility of a control
certication or assurance layer on top of the SAP instance. Conceptually,
this layer could be a part of SAP or an add-on, could be generic in
conguration or tailored to the instance, and could be re-thought as a
way to increase audit coverage as the original audit plan was applied in
an 18- to 24-month cycle, and under this design this layer would be
executed every day. Furthermore, the audit plan contained many
qualitative questions such as "Is there documentation for XYZ system?"
Elder et al. (2013) narrate a continuous monitoring effort at a large South
American bank in which internal audit monitored 18 different key
performance indicators (KPIs) for over 1400 branches of a bank. Daily
extracts of variances were obtained and, on a selective basis, followed up
by emails to the regional managers for the branches. These KPIs looked
to control overrides such as credit above allowable level or reversal of
certain types of transactions.
These examples illustrate (1) situations where auditors were in positions
of control over operational controls, which could result in a conict to the
auditor’s objectivity or independence and (2) that technology has
changed the needs, capabilities, and roles of the assurance function. As
suggested earlier, a more exible set of conceptualizations must evolve,
concerning auditor independence in particular. These examples are
focused on internal auditors, but a similar monitoring role could be
developed for external auditors and an ongoing monitoring opinion
could potentially be issued as a new CPA product.
Figure 1-2 describes the vision developed for multi-instances of ERPs and
an analytic engine supporting a set of functions. This view, however,
could be immediately after the event based on the two experiences
described above and would be an ex-post-facto overnight process, which
we would describe as retroactive close to the event meta-control or
assurance process.
5
Private notes Teeter, R.A., Warren, J.D., Brennan, R., and Vasarhelyi, M.A. 2007.
9
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 1-2: Pilot Continuous Monitoring of Business Processes at
Siemens, Rutgers CAR-Lab & Siemens Adding Intelligence (from
Alles et al, 2006)
Incorporating the concept of CCM into the original CA conceptualization
led to the renaming of the original CA to Continuous Data Audit (CDA)
where CA = CDA + CCM.
Continuous Risk Monitoring and Assessment (CRMA)
Vasarhelyi, Alles, and Williams (2010) suggested the addition of
Continuous Risk Monitoring and Assessment (CRMA) into the CA
schema where: CA = CDA + CCM + CRMA. CRMA is discussed in more
detail in essay 6, "Managing Risk and the Audit Process in a World of
Instantaneous Change" of this book. The essence of the CRMA concept is
displayed in gure 1-3 where risks are divided into three areas: (1)
operational, (2) environmental, and (3) black swans (Taleb, 2010). Black
swans are very remote risks with strong consequences that could arise, as
Taleb predicted the crisis of 2008. Risks are chosen judgmentally by the
audit team or management, and key risk indicators (KRIs) are associated
with the most important risks in each of the categories. The same basic
variance and acceptable variance model can be adapted to detecting
signicant changes of risk. The model can be parameterized at the initial
10
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
audit planning stage with heuristic or otherwise developed weights and
optimization procedures applied to determine an audit program. When
substantive changes in risk are perceived by the risk monitoring
procedures, the algorithm can be rerun, but management must also be
informed and joint action by assurance and management must follow.
This risk variance activation procedure also confounds the classical audit
theory, as many organizations have independent risk management areas
often broken down by type of risk or product. New conceptualization of
coordinated auditing or coordinated management, audit, and risk areas
must follow.
Figure 1-3: Structure for CRMA Effort
Continuous Compliance Monitoring
Very closely related to risk evaluation, and closely linked to the
increasingly regulated modern business world, is the area of compliance.
Although much of the traditional world of compliance is qualitative, it is
progressively being implemented by automated systems. Frequent
upgrades in ERPs, for example, at banks and insurance companies reect
the increased regulation, the need to reduce costs of compliance, and the
need to obey hundreds of regulations. In this essay, the development of a
compliance monitoring (COMO) approach to complement CA is
proposed.
11
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
The COMO approach would create comprehensive taxonomies of
compliance issues and progressive updates for regulatory changes
acknowledged by geography, area of activity, and the nature of
compliance rule (qualitative, quantitative, mixed, or other). It would
restate the CA equation to:
CA = CDA + CCM + CRMA + COMO
The integration of these views into a closer-to-the-event framework has
the advantage of improving assurance coordination, working towards
avoiding task repetition, and the potential usage of a conceptual and IT
platform. Table 1-2 illustrates one type of (quantitative) compliance
objective in relation to the topic of money laundering. As a caveat, if the
above functions are united into a joint conceptual view and one platform
implementation, the risks of their failure are much larger as a certain
degree of redundancy decreases risk but also increases costs.
Table 1-2: Example of Compliance Monitoring Table
Anti-money laundering
1. Compliance Topic: AML
2. Obligation or Compliance issue (for example, not to let over $10,000
through bank teller deposit without regulatory reporting)
3. Method of compliance: All transactions for a given deposit rule have
been captured and reported
4. Frequency capture daily, report quarterly
5. Importance: H M L HIGH
Compliance requirements can be largely qualitative, interpretive
especially of legal, regulatory requirements, but its fullment (for
example, fullment of the obligations) needs a degree of formalization in
measurement of supporting information, monitoring, and reporting.
Compliance fullment data is processed in the complex corporate legacy,
ERP, and other sources of big data where the company operates.
Traditional methods of extracting and evaluating an assertion of
fullment of compliance obligations to stakeholders and regulators are
anachronistic. Therefore the argument for continuous auditing applies to
compliance. Compliance management needs to be design-driven (for
example, formal structure for requirement denition, data capture, single
view of data bases, data visualization and interpretation from analytics
based representation). Continuous assurance and continuous compliance
assurance are complementary and can leverage many common design,
analytics, and technology components. Their integration is aimed to
12
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
alleviate the multiple problems generated by the proliferation of
audit-like organizations.
1.3 Guidance on Continuous Auditing
The rst guidance on continuous auditing was published jointly by the
CICA and AICPA (1999) and is often called the Red Book. This current
volume attempts to update the Red Book along several dimensions. Since
the publication of the Red Book, the Institute of Internal Auditors
published its GTAG 3 Continuous Auditing: Implications for Assurance,
Monitoring, and Risk Assessment (IIA, 2005) and ISACA its IT Audit and
Assurance Guidelines, G42, Continuous Assurance, (2010). In 2010, the
Australian Institute of Chartered Accountants also published its
Continuous Assurance for the Now Economy.
Leveraging this statutory work, continuous auditing literature reviews
(Brown et al, 2007; Chiu, Liu, & Vasarhelyi, 2014), and literature from
practice, this essay will summarize some basic theory postulates for
continuous assurance. Assurance, for purposes of this essay, is dened as
an umbrella of services that include the traditional audit and other services of a
similar or complementary nature that are emerging or being facilitated by new
technologies and business needs. (Vasarhelyi & Alles, 2006)
Considering the new assurance needs in control structure, control
compliance, data, and the existing guidance on continuous auditing, a
reconsideration and expansion of the elements in the concepts of
continuous assurance is needed.
2. THE ELEMENTS OF CONTINUOUS
ASSURANCE REVISITED
The advent of new information and analytic technologies has brought
about new products as well as new ways to perform business processes.
Since the early years of continuous auditing, business has substantially
evolved the continuous monitoring processes of production into many
other areas of activity including accounting and nance.
2.1 Continuous Auditing Versus Continuous
Monitoring
Considerable thought has been given to the problem of overlap between
management and assurance processes when they progress in the
automation route. KPMG (Littley and Costello, 2012) described it in
operational terms, as shown in table 1-3. Another approach would be to
13
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
consider some new type of conceptualization based on the new
economics of information, control, and risk.
Table 1-3: CA Versus CM
Continuous Auditing
Performed by Internal Audit
Continuous Monitoring
Responsibility of Management
•
Gain audit evidence more
effectively and efciently
•
Reactmoretimelytobusiness
risks
•
Leverage technology to perform
more efcient internal audits
•
Focus audits more specically
•
Help monitor compliance with
policies, procedures, and
regulations
•
Improve governance—aligning
business/compliance risk to
internal controls and remediation
•
Improve transparency and react
more timely to make better
day-to-day decisions
•
Strive to reduce cost of controls
and cost of testing/monitoring
•
Leverage technology to create
efciencies and opportunities for
performance improvements
Littley and Costello (2012), as shown in table 1-1 and the AT&T Bell
Laboratories development of Continuous Process Audit System (CPAS)
(Vasarhelyi & Halper, 1991) in parallel to management’s Prometheus
system (table 4) show a substantive overlap of management and
assurance analytics and the potential of the usage of similar systems to
support infrastructure. IBM’s
6
internal audit approach was to
commission three monitoring systems for auditees and progressively
obtain their agreement to use the system for monitoring by management.
Traditional audit thinking argues that if the auditor acts as a "monitorer,"
in one sense, he or she becomes part of the control system and loses
independence. On the other hand, the traditional audit can be viewed as
a form of tertiary control acting both as a deterrent as well as an
after-the-fact detective control. The progressively increasing set of layers
between the auditor and the data, as well as the massive nature of data
being used by large corporations, forces the existence of monitoring and
reporting layers, not to mention ERP software, web interfaces, legacy
systems, and outsourced processes.
Vasarhelyi & Halper (1991) initially developed the CPAS project aimed at
creating a meta-understanding of the system being audited and making
this system auditor-monitored. It became clear after a certain amount of
time that similar monitoring insight and analytics would be also of
interest to management and of benet in the utilization of the system
being monitored. Consequently AT&T developed the Prometheus system
(Vasarhelyi, Halper, & Esawa, 1995), which used the same technological
undercarriage of CPAS but with some unique analytics for both
6
As described in annual presentations at the World Continuous Auditing Symposium in Newark
(2011, 2012), that can be seen in http://raw.rutgers.edu/
14
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
management and auditing, as well as a larger common base of analytics
and monitoring controls.
Table 1-4 illustrates a series of reports, screens, and data monitoring
procedures based on AT&T’s RCAM system where there is examination
of data at multiple levels. While analytic 1 examines the overall
completion rate of the billing process, analytic 2 works at a much lower
and earlier level examining the data collected by the switches. Some
analytics are only provided to the audit functions, others are only of
interest to management monitoring, while others are to be supplied to
both. The CPAS conceptualization involved 4 major elements: (1) actuals,
(2) standards (models), (3) analytics and (4) alarms (alerts) in addition to
the method of measurement (direct data access or secondary capture).
Analytics in CPAS were provided in the form of formulae, rules, and, in
most of the instances, with graphic visualization.
Table 1-4: CA and CM at AT&T
7
Analytic
number
Process
CPAS (Continuous
Audit)
Prometheus
(Continuous
Monitoring)
1 Bill Completion
Monitoring
Percentage of bills
generated that were
completed
Percentage of bills
generated that
were completed
2 Calls recorded Long-term count of
calls adjusted for
cycle
Switch billing
integrity
comparisons
3 Bills missing Process integrity
reconciliation
Process integrity
reconciliation
4 Job sequencing in the
data center
Examination of
CA-7 and CA-11
reports
5 Discrimination of
reasons bills not printed
Staged counts
6 Specic Bill content
examination
Bill
images—content
extraction
summaries
For accuracy
verication
7 Bill sequencing controls For ctitious bill
detection
For production
monitoring
8 Continuity Equations For predictive
auditing (Kogan
et al, 2014;
Kuenkaikaew, 2013)
For error
detection and
process
monitoring
7
This table is illustrative in nature. It is loosely based on the actual experience of the monitoring
and assurance of the RCAM system in the 1986–1991 period.
15
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Kogan et al. (2014) applied the concept of continuity equations
expanding the original suggestion of Vasarhelyi and Halper (1991)
including the following:
r
Distinguishing exceptions from anomalies
r
Introducing time-lagged process measurements that reected
better the actual information ow in the system
r
Focusing on transaction-level monitoring with clarication of the
different levels of activities
r
Introducing the concept of automatic transaction correction into
the audit literature
Recent continuous auditing literature (Chiu, Liu, and Vasarhelyi, 2014)
has tried to improve the quality of the models that serve as the basic
elements of comparison for exception detection.
Table 1-5 compares and expands the original conceptualization of the
CPAS effort (Vasarhelyi & Halper, 1991; Halper, Snively, & Vasarhelyi
1988; Vasarhelyi, Halper & Esawa, 1995) with several research efforts
performed over the years.
Table 1-5: Expanding Conceptualization in CA/CM
8
Vasarhelyi & Halper
(1991),
Red Book (1999)
Expanded
Conceptualization
(1999–2014)
Notes
CPAS/Prometheus
effort
Several corporate
experimental
experiences
Work with P&G,
Siemens, Itau
Unibanco, and so
forth
Measuring Metrics Extractions from
many different
systems and
drawing from the
Big Data
environment
Great potential for
increased
validation of values
including database
to database
conrmations
Creating a
model
Standards Of comparison
Of variance
8
Highlighted items are expansions to the Vasarhelyi and Halper (1991) initial conceptualization.
16
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
Table 1-5: Expanding Conceptualization in CA/CM—continued
Vasarhelyi & Halper
(1991),
Red Book (1999)
Expanded
Conceptualization
(1999-2014)
Notes
Relating Analytics Representational
equations
Continuity
equations
Kogan et al., 2015
Visualization
Clustering and
transaction level
continuity
equations
For automatic fraud
detection and
transaction
correction
Alarms (4 levels)
Measurement Versus
Monitoring
Measurement
(indirect data
acquisition)
Direct data access
Introducing
external
comparative
benchmarks
Probabilistic data
relationships
Linking corporate
ERP data to big
data in the fringes
Dimension
Data Continuous data
auditing (CDA)
Vasarhelyi &
Halper 1991
Control Continuous Control
Monitoring (CCM)
Vasarhelyi, Halper
& Esawa, 1995;
Alles et al, 2006
Risk Risks (CRMA) Vasarhelyi, Alles, &
Williams, 2010;
Essay 6
Compliance Compliance (CM) Essay 1
2.2 The Elements of Continuous Audit
Vasarhelyi, Alles, and Williams (2010) have argued for the inclusion of
continuous risk monitoring and assessment (CRMA) in the CA schema:
"The audit planning process provides a template for how to make the
Continuous Assurance system dynamic: by formally incorporating into it
a risk assessment system that encompasses assessment of auditor
17
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
perceptions of risks and allocation of audit resources to risky areas of the
audit."
Vasarhelyi, et al. (2012) examined the continuous audit efforts of nine
large organizations. It was noteworthy that organizations had a series of
"audit-like" organizations (ALO) that competed for resources and
presented very different levels of technology use. In its principle 3.5, the
King report (Institute of Directors in Southern Africa, 1994, 2009) in
South Africa states that "The audit committee should ensure that a
combined assurance model is applied to provide a coordinated approach
to all assurance activities." A control and assurance automated
ecosystem can evolve the audit to create a more reliable and efcient
corporation.
All of the interviewed companies have a number of audit-like
organizations which perform assurance-like functions in different
areas. However, some of the audit areas overlap, and the results of
the review are not efciently shared among them as one manager
declared, "Let me start with my administrative boss. He is the director of
risk management for the organization. Underneath is internal audit.
Credit examination and our risk management/Sarbanes-Oxley...there is
another group that does testing that reports to Chief Legal Counsel.
Fraud is handled in our securities group, which is in our service
company. They perform investigations on internal and external
fraud...We do [received feedback], but not as much as we should."
One of the interviewed companies had up to seven ALOs, which resulted
in substantive differences in the quality of reviews, substantial
redundancy, lack of depth in the reviews, and what they called "audit
fatigue" where auditees would not cooperate due to the multiplicity of
assurance efforts. If the companies had continuous audit in stage 4, a full
continuous audit in stage 4, these problems could be eliminated as the
monitoring systems would be centralized and integrated. All ALOs could
share the systems and information, and their works would not overlap.
ALOs in this study included (1) internal audit, (2) compliance, (3) fraud,
(4) SOX, and (5) Basel, in most situations, although several other
nomenclatures and subdivisions existed. (Vasarhelyi et al, 2012).
18
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
Figure 1-4: Expanded Scope of CA including Compliance Monitoring:
An evolving continuous auditing framework
The original framework of continuous assurance can be expanded into
four elements: data, control, risk, or compliance. Figure 1-4 expands
Vasarhelyi, Alles, and Williams (2010) components to add an element of
compliance monitoring, expanding the scope of the CA and CM effort.
The same considerations of opacity of the data processing environment
and the difculty of access to its information apply to all elements of the
auditing framework that evolved since the AT&T CPAS effort.
3. INFORMATION TECHNOLOGY AND THE
AUDITOR
Traditional auditing has changed considerably as a result of changes in
IT, including more advanced ERP systems, increasing the use of on-line
transactions with both customers and suppliers, use of the cloud, and the
rapid expansion of data available for use by management and auditors.
The continuously evolving IT landscape leads to a variety of audit
challenges that compound over time, as summarized in table 1-6
(Adapted from Vasarhelyi and Halper, 1991).
19
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Table 1-6: The Evolution of IT and Associated Audit Challenges
(Adapted from Vasarhelyi & Halper, 1991)
9
Phase Period Evolution of IT Examples Audit Challenges
1 1945–1955 Input (I) Output
(O) Processing
(P)
Scientic and
military
applications
Data transcription
Repetitive
processing
2 1955–1965 I, O, P Storage (S) Magnetic tapes
Natural
applications
Data not visually
readable Data that
may be changed
without trace
3 1965–1975 I, O, P, S
Communication
(C )
Time-sharing
systems Disk
storage Expanded
operations support
Access to data
without physical
access
4 1975–1985 I, O, P, S, C
Databases (D)
Integrated
databases Decision
support systems
(decision aides)
Across-area
applications
Different physical
and logical data
layouts New
complexity layer
Decisions
impounded into
software
5 1986–1991 I, O, P, S, C, D
Workstations
(W)
Networks Decision
support systems
(non-expert) Mass
optical storage
Data distributed
among sites Large
quantities of data
Distributed
processing entities
Paperless data
sources
Interconnected
systems
6 1991–2000 I, O, P, S, C, D, W
Decisions (De)
Decision support
systems (expert)
Stochastic
decisions
impounded into IT
systems
7 2000–2010 I, O, P, S, C, D,
W, De,
Distributed (Di),
Distributed systems
Internet based
Cloud
Data stored in the
cloud and
replicated Virtual
IT software
8 2010–2020 I, O, P, S, C, D,
W, De, Di, Big
Data (BD)
Preponderance of
data that is
applicable in wide
array of business,
accounting,
accounting, and
auditing areas
Big data Multiple
sources of
automatic data
capture
9 2020+ I, O, P, S, C, D,
W, De, Di, BD,
Articial
Intelligence
Self-improving
systems Embedded
intelligent modules
Audit activities
and reporting are
slow and occur too
late
9
Highlighted items are expansions to the Vasarhelyi & Halper (1991) initial conceptualization.
20
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
For example, the challenges that emerged in phase 5 with the
decentralization and distribution of data were aggravated with the
advent of cloud computing in phase 7. The emergence in phase 8 of big
data (Vasarhelyi and Kogan, 2015; Moftt and Vasarhelyi; 2013) creates a
hybrid environment where systems must monitor the boundaries of the
broad external data environment, which is too voluminous to be
contained within the organization’s stores or its outsourced environment
(Krahel and Vasarhelyi, 2014). Organizations already scan and extract
from big data receptacles (for example, Twitter) and only retain selected
pieces or summaries. Although many systems exist that present some
degree of decision intelligence and even predictive behavior
(Kuenkaikaew, 2013), articial intelligence applications in business are
not yet so prevalent to create an audit challenge.
The evolution of IT also creates opportunities for the introduction of
further audit tools and methodologies especially as nancial systems
have moved towards decentralization, distribution, online posting,
continuous (or at least daily) closing of the books, and paperlessness
(Vasarhelyi and Yang, 1988).
The CCM application Alles, Brennan, Kogan, and Vasarhelyi (2006)
developed is much broader in scope than the Red Book denition, and
indeed, subverts its focus on only more timely audits. The point of CCM
is to exploit the very structure of the ERP system in order to bring about
automation, as opposed to simply doing the same audit procedures more
often. In their words, they were reengineering the audit process, not just
speeding it up.
Alles, Kogan, and Vasarhelyi (2003) proposed something similar when
they used the ability of ERP systems to propose the development of an
auditing "black box" that would enable the tertiary monitoring of the
audit itself. A decade later, a similar philosophy underlies the use by Jans,
Alles, and Vasarhelyi (2014) of event logs to audit business processes.
Alles and Gray (2012) state: "When analyzing the role of big data in
auditing it is critical to differentiate between whether what is meant is
more of the same kind of data that auditors are already using, or more
data of a different kind than what auditors have traditionally relied on to
give an audit opinion." The former approach would lead, for example, to
continuous auditing where the scope of data is not necessarily expanded,
but measurements are taken more frequently in time (Kogan, Alles,
Vasarhelyi, and Wu, 2014). By contrast, big data as it is dened below
pushes the domain of data far outwards from nancial data to
non-nancial data, from structured to unstructured data, and from inside
the organization to outside it.
21
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Over the last two decades, many new analytic and information
technologies have become ubiquitous. These technologies also have been
progressively applied to accounting and auditing. There have been
studies looking at the role of visualizations, exploratory data analysis,
process mining, tagging, the remote audit, predictive audits, and so
forth.
10
3.1 Evolving Database Audit Conceptualization
The core of traditional systems evolving from the early le-oriented
systems to hierarchical and today’s relational databases is the structured
nature of its data. Vasarhelyi and Halper (1991) pointed out levels of
audit complexity in their usage. Table 1-7, "Evolving Database Structures
and Their Audit" (expanded from Vasarhelyi and Halper, 1991), expands
their view with some of the new considerations of storage and data
provenance. Hierarchical data structures of the COBOL days were by and
large replaced by the relational databases that are the core of the modern
ERPs. With the ubiquity of the internet, there is the emergence of large
corpuses of unstructured data from which to draw expanded
information. A few facilitating axioms may be useful to introduce:
r
There are no reasonable limits of sources of data, but there are
great limits on what data an organization can actually store and
make useful.
r
In general data will tend to exist to support particular decisions or
processes, but the great challenge is to anticipate such needs and
create software and processes for its examination.
r
The costs of system development, improvement, and overlay obey
much different rules than the traditional xed and variable cost
managerial accounting model.
r
Many IT provisioning economic models are charged on an
incremental basis proportional to usage (Siegele, 2014).
Table 1-7 expands the table in Vasarhelyi and Halper (1991) with
additional system characteristics and presents the aforementioned
opportunities for the introduction of new tools and methodologies.
10
See http://raw.rutgers.edu/pcaob
22
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
Table 1-7: Evolving Database Structures and Their Audit (expanded
from Vasarhelyi & Halper, 1991)
11
System
Characteristic
Audit Complexity
(level 1)
Audit Complexity (level 2)
Database Documentation Data dictionary query
Database size User query Auditor query
Transaction ows Examine levels Capture sample transactions
Duplicates Sorting and listing Logical analysis and indexes
Field analysis Paper oriented Software based
Security issues Physical Access hierarchies
Restart & Recovery Plan analysis Direct access
Database interfaces Reconciliation Reconciliation and transaction
follow-through
Unstructured data Linkage to know
database elements
Establishment of stochastic
relationships between data
elements and unstructured data
Cloud storage Access and privacy
evaluation
Tests of system integrity and
business continuity
Big Data Selection of validating
parameters
Linkage to data streams and
extraction of meaning
Creation of new forms of
evidence
Integration of new evidence
into the traditional audit theory
(Hoogduin, Yoon, and Zhang,
2015)
3.2 Incremental Technological Change
The costs of more frequent assurance and its benets have substantively
changed with IT. In the 21st century information technology environment
(21CITE), the costs of performing processes has greatly changed with
advents in IT and networking as well as the reduction of the human labor
component. In essence the following has been noted:
1. Information storage and retrieval is being progressively
automated.
11
Highlighted items are expansions to the Vasarhelyi & Halper (1991) initial conceptualization.
23
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
2. The cost of creating a report that previously required incremental
labor per report now, once established, costs nothing to repeat and
is typically developed by the ERP developers.
3. With the modern systems, automatic data collection is changing
the schemata of data collection. Data from e-commerce
transactions, GPS
12
, and RFID
13
can be captured at dened time
intervals contingent on the business need being satised. (Moftt
and Vasarhelyi, 2013)
4. Cloud distribution and storage of created/sensed les creates
ubiquitous access and much more robust backup. Third party
sourcing creates several challenges on assurance but also some
degree of professionalism and competence in the data custody
function. (Mendelson et al. 2012)
5. A progressive incorporation of some forms of articial intelligence
into several business functions is creating a more stochastic and
judgment based set of decision rules. It cannot be assumed any
more that a well validated business procedure will respond
"correctly" as the rationale in the computer logic is a mix of
heuristic rules and complex analytics.
6. Robots are taking a larger and larger role in business processes
(Brynjolfsson and McAfee, 2014) and progressively systems with
articial intelligence will be integrated into the manual
performance of tasks.
7. The ubiquitous access to information and devices will also be of
great import. Two additional sources of internet connection—"The
Internet of Things" (Kopetz, 2011) and "Wearables" (Wei,
2014)—will provide further substantive data of particular value
for detective and preventive assurance.
These and many other considerations relative to technology and, most
importantly, to the economics of business processes are the drivers of
evolution on the continuous audit conceptualization.
3.3 The Audit Data Standard
Zhang et al. (2012) discuss the fact the audits are at risk of becoming less
relevant if they do not change to meet stakeholder needs, especially for
12
www8.garmin.com/learningcenter/training/oregon/
13
www.aimglobal.org/?page=rd_faq&hhSearchTerms=%22rd%22
24
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
timeliness and scope (for example, process assurance, data-level
assurance). Furthermore, they state that audit standards tend to lag
behind advances in technology, and many basic audit procedures have
not been updated to complement these developments. It also mentioned
that the Center for Audit Quality (CAQ) (2011) held roundtable
discussions that suggest that investors must act on timely and continuous
nancial information and it should be explored whether auditor
assurance should be provided for nancial information disclosed by
managers throughout the year.
Furthermore Zhang et al. (2012) argue that "auditors face a challenge in
accessing data as there are no standard requirements in place for data
availability. Auditors do not have ready access to their clients’ accounting
data, even when the clients’ business operations have become almost
entirely digitized. As technology is the major driver of the evolution of
the audit process, the AICPA Assurance Services Executive Committee
(ASEC) Emerging Assurance Technologies Task Force is trying to pave
the way for enhanced use of technology and advanced data analytics in
the audit process. The audit data standards, including data standards,
data access, audit applications and continuous audit, are formulated to
facilitate data acquisition in a standardized fashion and advance the
process of audit automation" (Vasarhelyi et al. 2011).
The CAQ initiated an effort to guide the profession towards a set of audit
data standards that would guide organizations to make data available to
auditors in a standardized format allowing for the homogenization of
utilization of data using common auditor oriented applications ("apps").
The AICPA’s ASEC took this effort over
14
and is progressively issuing
this guidance. Figure 1-5 displays a symbolic view of an automated audit
architecture that links: 1) existing corporate IT systems (including
outsourced ones and Big Data Links, 2) extractor routines, 3) ADS
standards, 4) automatic audit plan generation, 5) apps, 6) app selection
routines, and 7) continuous assurance.
14
www.aicpa.org/InterestAreas/FRC/AssuranceAdvisoryServices/Pages/AuditDataStandard
WorkingGroup.aspx
25
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 1-5: ADS Ecosystem Architecture
Zhang et al. (2012) stress that the Audit Data Standard project is an effort
to bridge the gap (Kaplan, 2011) between accounting scholarship and
practice. "Kaplan (2011) argues that accounting faculty, as scholars in
professional schools, have conducted studies that are mostly reactive and
put overemphasis on the existing practice instead of advancing the
practice. He suggests that accounting scholars should ll the void in
academic research and focus on developing knowledge for leading edge
practice. The emergence of data standards and audit applications (Apps)
is the fruit of academic and practice cooperation."
The AICPA issued initial guidance on the Audit Data Standard in 2013,
which included the creation of the following audit data standards: (1)
base standard, (2) general ledger standard, and (3) accounts receivable
subledger standard. The work continues in extending the standard to
other cycles and directions. Currently underway are order-to-cash and
procure-to-pay subledger standards.
4. THE NEW CONTINUOUS AUDIT
New considerations in continuous audit tie closely to the evolution of
information and analytic technologies that grandly expanded the feasible
set of monitoring and assurance activities. Many of these activities that
could be extremely benecial are not performed either because of
26
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
ignorance of their availability, misconception of their nature,
misunderstanding of the costs, or mainly because of the serious costs that
may occur in such a litigious society as the United States if substantive
corporate reporting problems are detected. Many of the instances where
a "material error" was detected, the problem had existed for years in an
increasing scope. The problem tends to explode when the adverse
business economics that usually causes misrepresentation is too large to
be unnoticeable.
Figure 1-6 lists the dimensions of the assurance process that are evolving
in the new continuous audit: (1) assurance level, (2) time focus, (3) time
interval, (4) data source, (5) chosen procedure, (6) choice of assertion, (7)
analytic methods, and (8) assurance entity. Other dimensions may also be
of importance in the progressive evolution of audit theory over time.
Figure 1-6: Dimensions of the New Continuous Audit
Halper and Vasarhelyi (1991) recognized the evolving nature of
information technology table 1-6, "The Evolution of IT and Associated
Audit Challenges" (Adapted from Vasarhelyi and Halper, 1991), and its
opportunities in relation to assurance. Here the discussion is expanded to
27
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
look at several of the evolving dimensions transforming the panorama of
audit (internal and external), control, and management.
The concept and practice of internal control evaluation (design and
compliance) has been in the literature for many decades. Sarbanes Oxley
expanded its formalization by requiring auditor assurance on
management assertions about internal controls. Although the literature
of data audits and its methods have evolved for many decades, research
on internal control representation formalization (Cash, Bailey, and
Whinstone, 1977; Bailey and Meservy, 1986; Bailey et al., 1985) has been
sparse. The issues of control representation, assessment, congurable
controls, compliance, and verication are to emerge as a major need for
professional work and research. The monitoring of controls, the effect of
their modication by tailored ERPs or overrides in congurations, and
the existence of tens of thousands of controls plus compliance
requirements creates a very complex environment both for management
and assurers.
4.1 Assurance Level
Kogan et al. (2014) focus on transaction level assurance whereby
continuity equations are used to monitor transactions through the stages
of a hospital supply chain. It utilizes the patterns of delay between
processes to improve predictions and to perform automatic transaction
correction. It improves the basic quality of data and allows for preventive
auditing and automatic transaction correction.
Control level assurance (CCM) has partially replaced the traditional
process of internal control evaluation and compliance testing. The ERP
environments with pre-set controls have already demonstrated a reliable
information structure, but new issues such as congurable controls have
appeared to concern management and assurers.
Account level accuracy can be supported and assured at many low,
intermediate, and high levels of accuracy. Dashboards (Moharram, 2014)
and visualizations (Alawadhi, 2014) are improved with new technology
which combines analytic transformation and takes advantage of the
attributes of human information process. With the evolution of
technology, it is possible to develop and test assertions at a much ner
and directed manner.
Statement level assurance allows for combined assessment of accuracy,
taking into consideration transaction accuracy, control climate, and all
levels of account level accuracy. Each level of assurance actually serves
different purposes for both management and auditors.
28
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
4.2 Time Focus
Auditing has been retroactive since its inception. Auditors examined past
accounts for accuracy and reported perceived discrepancies. Figure 1-7
illustrates that auditing can both be reactive and predictive
(Kuenkaikaew, 2013). When predictive, the auditor (Vasarhelyi and
Halper, 1991; Vasarhelyi, Alles & Williams, 2010) will rely on models
(standards) to predict results (performance) in an account (transaction)
(Kogan, et al., 2014). This prediction is compared with actuals in near-real
time to detect substantive variances in monitored processes. Much of the
recent research on CA has recently focused on developing better models
for actual comparisons (Chiu, Liu, and Vasarhelyi, 2014; Brown et al.,
2007). These variances, from improved models, are treated either as an
alert to the management and audit function or, if the system has reliable
lters, to prevent faults from progressing toward execution. Modern
systems combine management action and assurance. Much conceptual
work is needed in the re-denition of concepts such as auditor
skepticism, independence, materiality, auditor role, audit objectives, and
so on. Many of these needs are motivated by the ever-increasing level of
automation in corporate business systems and the correspondingly
automated nature of tools used by individuals. The advent of a
progressively bring-your-own-device (BYOD) environment (Loras et al.,
2014) is affecting the locus of the control and assurance. Some of the
BYOD tools like smart phones already incorporate predictive algorithms
to perform a set of integrative functions for the user. These functions
associate typical behavior with data integration to decrease key-strokes
by the user. For example if the device detects a request for contact and a
telephone number or an address, it may immediately prompt a call or a
map to the location.
Auditors will eventually have predictive procedures to drive them to
data level prediction (Kuenkaikaew, 2013), procedural prediction (based
on the experience of other auditors using the tools and maybe the
guidance of the audit plan), and control prediction (where weaknesses in
controls or process changes drive the activation or re-parameterization of
controls.)
Intelligent preventive controls are progressively permeating the corporate
IT ecosystem and personal devices. The relationships between processes
that have always existed may now be explored analytically and visually
for management and assurance purposes. If the predictive ability of
models is high and processes modularized and discrete, it may be
possible to prevent an error, automatically correct an error, or correct a
control deciency prior to its occurrence. For example an insurance
company develops a forensic model to determine if a particular claim
29
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
payment is inaccurate (fraudulent or in error). This model is very
accurate in generating a number of false positives and false negatives. It
can develop a process that once a transaction is ready to order a wire
transfer, it is subjected to the same forensic model and, if the level of
condence of accuracy of the transaction (the loading function for the
transaction to be further discussed later in this essay) is below a certain
threshold, the transaction is blocked and a group of auditors (Elder et al.,
2013) proceeds to examine it and release it or not. The economics for this
preventive behavior depends on the amounts of the electronic fund
transfers, the incidence of erroneous transactions, the losses/costs
historically incurred in these (detected and undetected but estimated)
errors, and the cost for an auditor or manager to perform this supervisory
and assurance action.
Figure 1-7: Time Focus of the Audit Methodology
4.3 Time Interval
The original CA work aimed at using the evolution of technology to
replace the work on the annual audits, but the client organization was
internal audit. It rapidly became clear that external audit rms do not use
CA techniques but consult with internal audit departments on the matter.
(See essay 2 in this book.)
As the technological drivers of Continuous Assurance continue to
rapidly progress, it has proven difcult to reach consensus on what
30
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
Continuous Assurance actually encompasses. There is the need to
update the AICPA and CICA denition of continuous assurance to
do away with written audit reports, which are redundant in
today’s world of electronic communication. Even more
importantly, the word ‘continuous’ undoubtedly would not be
used today, because it implies a frequency of auditing that is both
difcult to achieve technically without impacting the operations of
the entity’s IT systems, and probably beyond the needs of most
users. The different elements of a corporate information system
have different pulses and natural rhythms. The assurance process
must be coherent with these rhythms to be useful and effective.
(Adapted from Vasarhelyi, Alles, and Williams; 2010.)
This new view of CA, encompassed in this essay, disagrees with the
above statement that the "frequency of auditing that is both difcult to
achieve technically without impacting the operations of the entity’s IT
systems, and probably beyond the needs of most users." Technology is
already present to achieve "close to real time assurance." Corporate
business ecosystems will be by nature distributed, real-time, and most of
all very opaque to the naked eye. Consequently there will be many
systems that will be difcult to audit unless a transaction is monitored
frequently, predicted in value, prevented if deemed probably erroneous,
and so forth. The nomenclature (is this management, control, or
auditing?) given to the meta-control and assurance function is of less
import than its progression over time and the integrated systems need.
Assurance close to the event allows for inter-process fault blocking and
rapid management/auditor intervention into incorrect or unexpected
events, which is one factor that was not to be considered in the traditional
audit approach.
4.4 Data Source
The new corporate data presents a wider scenario of data sources (Krahel
and Vasarhelyi, 2014) internal (endogenous) from ERPs, legacy systems,
web-facing systems, and middleware. This data is complemented by
associated (outsourced) systems and by bridges to external (exogenous)
data of several origins. Data can come from public databases (for
example, macroeconomic data, market data such as Compustat and
CRSP), from bridges to the larger data environments of video, text, and
audio (Moftt and Vasarhelyi, 2013), and from the many automatic data
collection devices that are emerging for multiple purposes. See gure 1-8
for further examples.
31
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 1-8: Sources of Data (adapted from Qi and Vasarhelyi, 2014)
4.5 Chosen Procedure
Audit procedures have been frequently formalized under GAAS in order
to create guidelines for verication of nancial statements. Unfortunately
the standards have not yet explicitly embraced more advanced
technological methods that can deal with the emerging challenges of big
data, cloud computing, embedded decision making, and the like. In
general the audit standards allow for evolution of procedures but do not
necessarily facilitate or require such an effort. See table 1-8 for procedures
and their evolution.
Table 1-8: Procedures and Their Evolution
Traditional procedures Modern procedure
Client acceptance
and investigation
Multiple mainly manual
methods including
investigators
Identical plus extensive text
mining of sources like
newsprint and social media
Client monitoring Extensive text mining of
sources like newsprint and
social media
Population estimate Statistical or judgmental
sampling
Big data population
estimation
Full population
measurement
32
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
Table 1-8: Procedures and Their Evolution—continued
Traditional procedures Modern procedure
Conrmation Manual conrmations or
conrmation.com
Close to full population
conrmation with database
to database conrmatory
pings/handshakes
Substantive testing Manual examination of
documents
Electronic documents,
process mining
Analytical review Comparison of end of
the month ratios and
their trends
A wide selection of analytics
procedures at most stages of
the audit
Internal control
evaluation and
compliance testing
Manual tracing,
observation, structural
evaluation
Reliance on ERP design,
CCM
4.6 Choice of Assertion
International auditing standards and U.S. GAAS classify assertions into
three categories:
r
Assertions about classes of transactions and events for the period
under audit
r
Assertions about account balances at period end
r
Assertions about presentation and disclosure
To which we add the following:
r
Assertions about emerging issues of less traditional nature
An assertion basically represents the concern of auditors of particular
system faults. Exploratory Data Analysis (EDA) (Liu, 2014) allows for
preliminary data examination leading to choice of assertions to be
considered in a particular audit. By looking at the data characteristics and
distributions and contingencies, the auditor will start with basic
assertions and choose additional ones to be considered. EDA will allow
for the creation of assertions and the transformation of EDA into
conrmatory data analysis.
4.7 Analytic Method
The development of new IT infrastructure, analytic methods, and the
expansion of ALOs is changing the potential of continuous audit to a new
dimension described in table 1-5: Expanding Conceptualization in CA
and CM. The essence of audit automation and the progressive evolution
of an audit ecosystem entails synergistic integration of its elements. As
has repeatedly been discussed in this essay, systems that support
33
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
business processes have become too complex to be efciently addressed
through pure human assurance. Layers of data, software, and the
interconnection with upstream and downstream systems (and processes)
make observation and evaluation very complex.
In general an entirely new family of audit analytics is emerging
15
that can
affect all parts of the audit engagement and can allow the use of an
expanded data framework that includes external big data to support
audit assertions in an unorthodox manner. Table 1-9 illustrates the
number of potential changes and improvements to assurance
methodologies. It should be considered together with table 1-10 in which
the emphasis is more on procedures.
Table 1-9: Audit Phases and Analytic Methods (modied schema of
Cushing and Loebbecke, 1986)
Audit phase
Applicable analytic
methods
Observations
Client
examination
•
News media
monitoring
•
Social media
monitoring
A large set of sources allows for
environmental scanning of events
with directors, their reputation,
the behavior of competitors, and
events in the industry
Audit
Planning
•
Ex-ante risk assessment
alaCRMA
•
Ratio analysis
Peer industry group evaluation for
performance
Audit risk
assessment
•
CRMA
The "material" change in the risk
situation requires changes in
continuous monitoring,
management action, and in
continuous audit parameters
Internal
Control
evaluation
•
Process mining
•
Analytical modeling
Much reliance on the "best of
class" nature of designed ERP
systems but hampered by the fact
that most large organizations’ data
is a mix of ERP based and many
other sources
Compliance
testing
•
Process mining
•
CCM
Concern about user congurable
controls requires monitoring these
settings through a CCM
methodology
15
http://raw.rutgers.edu/audit_analytics_certicate
34
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
Table 1-9: Audit Phases and Analytic Methods (modied schema of
Cushing and Loebbecke, 1986)—continued
Audit phase
Applicable analytic
methods
Observations
Substantive
testing
•
Cluster analysis
•
Database-to-database
conrmations
•
Continuity equations
The emergence of very large
number of transactions, the ability
to store them online, the reliance
on electronic documents and
records, and the usage of XML
derivative languages to exchange
data from upstream and to
downstream systems changed
drastically the items to be tested
and requires new audit tests that
are not yet in the vernacular
Opinion
formulation
Formal expert systems for
the evaluation of new
forms of audit evidence
Systems for estimating
potential for audit failure
based on internal evidence
and exogenous variables
With the multitude of data forms
and volume, and the lack of direct
observability of data, audit
systems will have to be
substantially automated with a
symbiotic process of opinion
formulation partially relying on
machine observation and opinion
formulation
4.8 Assurance Entity
Different ALOs have a mix of complementary, independent, and
overlapping objectives. Assurance coordination, as recommended to be
implemented in the King report, must take into consideration the
evolving variables discussed in this section: (1) assurance level, (2) time
focus, (3) data source, (4) chosen procedure, (5) chosen assertion, (6)
analytic method, and (7) the specic issues and objectives of the different
assurance entities. Organizing a matrix of the above variables, ALOs, and
technology platforms can help to create a more efcient assurance
function.
5. QUESTIONS REGARDING SOME AUDITING
CONCEPTS IN THE MODERN ENVIRONMENT
The speed of technological change is overtaking the ability of business to
change and of the multiple lines of defense. The inherent opacity of the
layers of technology opens exposures at the same time that it creates
35
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
capabilities for business. The same technology that allowed data to be
processed rapidly and consistently also allows for consistent errors and
their distribution without human observation. The same technology that
allows for remote access of computers allows for foreign intrusion and
virus diffusion. The same technology that facilitates electronic
transactions with credit card magnetic information also allows massive
and intrusive capture and leakage of credit card information at reputable
organizations such as Target
16
and Home Depot
17
. As earlier discussed,
the roles of management, internal audit, and external audit are
overlapping and use the same tools. Figure 1-10 attempts to integrate
some high-level functions that will compose some of the elements of
future management and assurance. Prior to its discussion some basic
issues in modern assurance are discussed including: (1) progressive
implementation of assurance systems, (2) functional migration of roles
and tasks, (3) concepts to be evolved in the new audit conceptualization.
5.1 Stochastic Opinion Rendering in a World
of Statistics
The nearly "yes" or "no" nature of external audit reporting doesn’t
provide the types of insights or commentary that stakeholders may nd
informative. The audit literature has proposed over the years several
forms of probabilistic reporting and more explanatory audit opinions.
These would give more information to stakeholders, but in general the
proposed methods are limited.
Associated with the concept of probabilistic reporting, the modern audit
could benet from a real-time auditor dashboard. The issues related to
legal liability, stakeholder needs, and the natural reticence to change will
tend to make this evolution challenging. However several commercial
products and research efforts are developing these dashboards in internal
audit organizations responding to real needs of system monitoring.
Internal audits would provide additional value with the issuance of
probabilistic reporting.
In general, materiality estimates relate to dollar value in relation to a
value on a nancial statement. For example, 5 percent of net income is
compared with the total value of the account on an account-by-account
basis. The audit literature has been linked to the concept of materiality
for a long time. Clearly there are decreasing returns in the economics of
16
www.businessweek.com/articles/2014-03-13/target-missed-alarms-in-epic-hack-of-credit-
card-data
17
www.reuters.com/article/2014/09/09/us-usa-home-depot-databreach-
idUSKBN0H327E20140909
36
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
data evaluation and review. In the engineering sciences the concept of
relative and acceptable errors are common. Unfortunately there are no
precise denitions of materiality in the auditing standards literature
(Elliott, 1986). Furthermore, information technology has changed the cost
structure of both the benets of an audit as well as the costs of performing
audits by making information storage and retrieval very different.
The new environment changes the costs and benets of assurance. Source
documents are indexed and electronic. Analysis activities can be mainly
automated. A wide net of automatic document reviews can be
communicated to staff and serve as a serious deterrent to malfeasance. If
auditor substantive processes can be formalized and support systems
evolve towards all electronic processes, full population evaluations may
be possible and desirable depending of a set of very different cost-benet
tradeoffs.
A new conceptualization of materiality may be needed now with
different considerations of dimensions such as monetary value, volume
of transactions, type of usage, and probability of outcome. Furthermore,
for the audit to be more informative, it may be desirable to disclose more
details of relative expected error and the auditor may create a product
that provides a more detailed set of relative error assessments.
Furthermore, there are qualitative and quantitative aspects in audit
decision making, as many of the analytic-based monitoring processes
will be out of the eyesight of the auditors, there must serious thought
given to automatically bringing relevant qualitative evidence to auditors.
5.2 New Audit Products
The creation of new digital products has faced a Cambrian moment
(Siegele, 2014) of dramatic change where the cost characteristics of
e-products (mainly xed cost and very low marginal variable costs) are
being reected by the method of provisioning and charging for new
products. Auditors need to develop layered monitoring systems with
embedded elements such as sensors (for example, RFID, GPS, computer
vision), analytic intelligence, and exception detecting and rerouting
capabilities in order to provide additional assurance services.
Table 1-10 expands the conceptualization of the audit opinion and table
1-11 adds features that could be parts of the nature of the product.
Clearly, unintended consequences and the legal environment would
permeate the world of expanded assurances.
37
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Table 1-10: Expanded Opinion Conceptualization
We have The For the period And we found
Examined Financial
statements
Year Materially correct
Monitored Account Month Reliable to the 99%
level
Analyzed Transactions Continuously
or close to the
event or in the
appropriate
frequency
The enclosed
exceptions for the
period
Prepared Controls The following alerts in
the attached URL
Reported Process Correct with an
acceptable error rate
of 1%
Reported and
veried
Outsourced process
Shared
examinations
Automated
decision settings
The settings to be
adequate to perform
the continuous
assurance function
Security of user
information such as
social security
numbers and
passwords
The system vulnerable
to serious attack
5.3 Management, Control, Assurance, and
Other Meta-Processes Confusion of Concepts
It may be overambitious to attempt to resolve the confusion generated by
the expansion of functions taken over by technology and their effect on
the "lines of defense" discussed earlier in this essay. It sufces to
understand that internal and external business related functions aim to
achieve corporate objectives. The nature of the objective, the
characteristic of job functions, the type of technology progressively being
used, and the nature of the contractual relationship with vendors,
assurers, suppliers, and customers will affect several management
controls and assurance functions. The historically evolved set of rules
and regulations that permeate the environment rely on denitions that
may not be relevant in this age of automation and piggybacking (Siegele,
2014) of technologies and processes. Some examples of concept confusion
include the following:
38
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
1. If a business has an audit group that reviews and decides on
alarms found (Elder et al., 2013), are they performing an audit or a
management function? By doing this are they losing their
independence but as internal auditors still maintaining their
objectivity?
2. If the auditor intervenes in the process when a ag arises, is
he/she losing independence but as an internal auditor still
maintaining objectivity?
3. If a system ags thousands of exceptions and only the "exceptional
exceptions" are being examined by auditors, is this lack of due
diligence?
5.4 Independence
Sarbanes Oxley required CPA rms not to perform a wide variety of
consulting services for their clients. At rst blush, this seemed a good
step in light of the egregious aberrations of the Enron and WorldCom
nature where the perception was the large audit fees paid for system
services to the client blurred the vision and integrity of external auditors.
Likely the need for understanding large systems, partnership with
internal organizations, and a dramatic set of environment-changing
events
18
may change the view of independence impairment and may
revert to some degree of auto-policing and the redenition of
independence conict. This statement is not aimed to really discuss
independence, but it is an illustration of changing conditions that may
change concepts in management function as well as the migration of
functions to automation and their consequences on organizations,
regulations, and social matters.
5.5 Migration of Functions to Automation
The original applications of computers focused on facilitating intensively
computational tasks such as the calculation of trajectory tables for
cannons in warfare, a task that was being performed manually by a large
number of soldiers (Fishman, 1982). With the introduction of magnetic
tapes into computer systems, and their sequential data organization, the
business purpose of computers became obvious and hundreds of
employees manually preparing utility bills were let go and replaced by
massive process automation. Once the very obvious large labor
replacement tasks were accomplished, demonstrating the economic
benets of automation became more complex. Typically IT solutions at a
18
Such as breaches in computer systems, cross country mergers, substantive integration of machine
intelligence into decision making processes, and the integration of robots into corporate production
processes.
39
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
more advanced stage improve data quality and processes but are not
very closely tied to labor replacement. One of the key lessons from
decades of IT and now analytic technology implementation is that to
achieve the real benets of substantially changed technology, processes
much be rethought and reengineered (Hammer, 1990; Davenport, 1992,
O’Leary, 2000).
Table 1-11: Imagined Automation, Migration of Functions,
Technologies, New Processes and Methods
Technology Automation
Migration of
functions
New processes
RFID Of inventory counts
Verication of retail
sales Verication or
warehouse
deliveries
Overlap between
management,
control and
assurance
Inventory counts,
inventory tracking,
sales, purchases
GPS Of payroll
validation Of travel
expenses
Employee work
location and
existence
conrmation
Dashboards Audit by exception
(ABE) Audit plans
are complemented
by exception
activators
Monitoring of
alerts, macro
process indicators
Auditor close to the
event examination of
perceived alerts
Cloud
storage
Group based
work-papers
Some work-paper
functionality goes
to audit black
boxes
Some sharing of
auditor les and
black boxes between
management and
auditors
Big data Process integrity
monitoring is
included in the
audit process
Bots are integrated
into process ows
insteadofhuman
intervention
Creation of
monitoring functions
relating big data
variables and
assurance
Clustering Automatic outlier
detection processes
are incorporated
into the ecosystem
Outlier cluster
measurements are
automated
Continuity
equations
Process efciencies
are measured
through
inter-process
equations
Process
relationship
equations are
created, disclosed,
andusedfor
monitoring
40
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
Table 1-11: Imagined Automation, Migration of Functions,
Technologies, New Processes and Methods—continued
Technology Automation
Migration of
functions
New processes
Machine
learning
techniques
Predictive/
preventive audits
facilitated by better
predictions.
Predictive
technology further
expands audit by
exceptions
Process
mining
Automatic
transaction path
analysis and
monitoring is
implemented
Text mining Of client acceptance
and engagement
renewal
Continuous client
investigation
examining
news-pieces and
social media
Conrmations Database to
database
population and
value resolution
Essay 4, "Reimagining Auditing in a Wired World" illustrates the blue sky
scenario of a potential imaginary future audit. The ensuing ctitious
vignette illustrates the potential prospective evolution of audit
automation—progressively embracing different technologies and
automating business processes, control methods, and its assurance layer
and processes.
AIC auditors serve a large clientele mainly focusing on retailers. In order
to improve its efciencies over the years, AIC has implemented a series of
changes in its technological capacity and methods of assurance. Its
relation with CL Grocers (CLG) illustrates this fact.
r
AIC convinced CLG to make agreements with its larger suppliers,
banks, and clients to adopt a transaction and account level
conrmatory protocol where, at pre-established intervals, CLG
and its partners exchange conrmatory pings. A dashboard
manages this process, which is shared between AIC and CLG,
although with different reports.
41
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
r
AIC runs on frequent basis text protocols examining social
networks and news pieces for items relevant to CLG, its
competitors, directors, managers, and employees.
r
AIC has by and large changed to a risk-based audit by exception
methodology whereby risk monitoring encompasses external and
internal factors and the assurance effort coordinates with
management.
r
AIC has adopted a commercial system of automated working
papers that track auditor keystrokes, phone communications, and
several embedded modules in the client system on a constant basis.
r
AIC and CLG cooperate on ne tuning a system of predictive
analytics that creates forecasts for key accounts and processes of
CLG. These are used for process monitoring, preventive
monitoring, and for some of CLG’s communications with its
stakeholders.
r
AIC’s staff has a wide variety of skills, in particular IT and
analytics, and has a very intensive lifelong training program. AIC
also monitors its staff through external and internal information
sources.
r
Larger inventory items have RFID chips and their movement is
recorded through the supply chain with the participation of
external partners.
r
AIC has a wide menu of assurance and advisory services it offers
and it contracts not only with CLG but also many of its partners
for services such as covenant monitoring, asset existence, process
monitoring, nancial statement assurance, and so on. The
compensation for these services is mainly parameterized on the
characteristics of CLG’s business, not labor hours. AIC will also
perform compensated work for the government relative to tax,
ecology, and labor practices. The coordinated audit has many
partners.
New protocols, technologies, and standards must cooperate in order to
achieve a progressive layering and coordination of management, control,
risk, and assurance functions. The following section discusses a symbolic
view of what the audit ecosystem would entail.
5.6 The Audit Ecosystem
Businesses are now often described as ecosystems. A logistic supply
chain is managed by a multitude of information ows, actors, and IT
infrastructures within an evolving timeframe. The Economist described
practical ecosystems:
42
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
Pioneers such as Amazon have built cloud-based "ecosystems"
that make content such as its electronic books widely available.
Even though the rm has its own e-reader, the Kindle, and has
hatched a tablet computer too, it has also created apps and other
software that let people get at their digital stuff on all sorts of
devices, including PCs.
Other companies are developing their own ecosystems in a bid to
make people’s mobile-computing experience even more seamless.
Google’s recent $12.5 billion acquisition of Motorola Mobility,
which makes smartphones, tablets, and other gadgets, will enable
it to produce a new crop of devices to show off its cloud services,
such as Gmail and Google Docs, to best effect. Apple is stepping
up its integration efforts, rolling out an "iCloud" in which people
can store up to 5GB of content for nothing, and more if they pay.
(Economist, Nov. 4, 2010)
Figure 1-9 represents a potential schemata for an audit ecosystem with a
set of elements aimed at dealing with the emerging 21st century
information technology environment (21CITA) (Kozlovski and
Vasarhelyi, 2014).
Its main elements include the following:
r
Examination of transactions and account levels at their entry point
in the system, typically with process evaluation apps looking for a
variety of generic problems with transactions such as incomplete
or incoherent data, high loadings in potential fault discriminant
functions, data out of the normal transaction stream, and so on.
r
Examination of transactions / account levels using time-series,
cross-sectional, and time-series cross-sectional analyses to detect
aberrant transactions on a comparative and historical trend basis.
r
Constant monitoring of the environment through soft bridges with
social media, news pieces, competitor monitoring, and so on.
r
Development and monitoring of mixed loading factor equations
for exception detection.
r
Large audit databases aimed at validation of daily feeds and
collection of account-level data for cross-sectional analytics.
r
Audit plans that are sensitive to risk levels and variations. The
audit plan in a real time audit environment has to be adaptive
contingent on changing conditions and rely on continuous
monitoring of transactions (and adjustments) entering the system
as well as monitoring the time series and cross-sectional trends.
r
Hundreds or thousands of apps available in the environment
respond by creating tests with the dynamic adaptation of
assertions.
43
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
r
Many of the apps would be autonomous agents either time
activated (krons), circumstance activated (daemons), or audit plan
activated.
Kozlovski and Vasarhelyi (2014) discuss agents in an audit context as
follows:
The various agents presented by Papazoglou (2001) for use in a
digital ecosystem are also applicable to an audit ecosystem:
r
Application agents: CA/CM agents that are specialized to a
single area of expertise and work in cooperation with other
agents to solve complex audit problems are but one example
of the many application agents that encompass an audit
ecosystem
r
Personal: (or interface) agent: Work directly with users,
primarily client and provider staff, to help support the
presentation, organization, requests, and information
collections, such as providing user access to audit results
r
General business activity agents: Perform a large number of
general support activities such as search agents that navigate
effectively through fragmented online electronic information
in order to provide guidance to the CA/CM agents
— Information brokering agents: Provide facilities such
as locating information on Web sources or other
agents that are required to solve a common problem,
such as specialized agents to support CA/CM agents
in addressing data anomalies, for example
— Negotiation and contracting agents: Negotiate the
terms of a business transaction as regards to
exchange and payment, as is required when
transacting for audit services
r
System-level support agents: Provides objects with access not
only to other application objects but also to such facilities as
transaction processing when acquiring audit services
— Planning and scheduling agents: a multi-agent plan
is formed that species the future actions and
interactions for each agent. Typically, an agent may
act as the group planner for a cluster of agents
surrounding an application agent such as to support
multiple CA/CM agents analyzing big data
simultaneously, for example
44
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
— Interoperation agents: Audit processes may require
accessing information from legacy systems and
CA/CM agents from separate providers
— Business transaction agents: Can be used to
determine new CA/CM product offerings to
incorporate in the audit ecosystem
— Security agents: Provide security measures for
information, communications and data to or from
the audit ecosystem (Based on Papazoglou 2001).
Kozlovski and Vasarhelyi (2014) also discuss the characteristics of an
audit ecosystem in gure 1-9. It represents the many characteristics of an
audit ecosystem in a single view including attributes, features, and
software agents. The schema presented in gure 1-10 complements gure
1-9 as it focuses on the dynamics of transaction processing, rather than on
detailed characteristics.
Figure 1-9: Audit Ecosystem Characteristics (Kozlovski and Vasarhelyi,
2014)
45
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 1-10: The Audit Ecosystem
The 21CITE promises different levels of integration between the
organization and its data environment. The data sources to be scrutinized
closer are in internal data, and often outsourced data requires reliance on
a third party (the auditor of the outsourcer). As experiences with viruses
and control structures, new forms of technology, analytic methods, and
human inventiveness constantly change the panorama, new forms of
fraud, as well as weaknesses in software, are constantly appearing and
must be considered.
6. CONCLUSIONS
The rapidly accelerating pace of technological change has created a social
drag where socioeconomic systems hold back technological progress. The
ubiquity of computers in the performance of business processes brings
the need for strict formalization of legal and business rules (Krahel, 2011)
and automation has also resulted in a change in economics. This essay
sets the groundwork for the evolution of continuous assurance initially
formulated by Groomer and Murthy (1989) and Vasarhelyi and Halper
(1991) and divulged by the publication of the CICA and AICPA
continuous auditing guidance (Red Book, 1999) later supplemented by
the IIA (2003) and ISACA (2010).
46
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
The early work on CA focused on using the benets of automation to
perform a more frequent and deeper audit. This essay emphasized a
wider frame of thought by considering the effects of technological change
on business and the role of a more continuous form of assurance, with
different economics, conditions, and processes than are used today.
In this new environment there are no reasonable limits of sources of data,
but there are great limits on what data an organization can actually store
and make useful. Data will tend to exist to support particular decisions or
processes, but the great challenge is to anticipate needs and create
software and processes for its examination. The costs of system
development, improvement, and overlay obey much different rules than
the traditional xed and variable cost managerial accounting model. The
fact that many IT provisioning economic models are charged on an
incremental basis proportional to usage will change the profession’s
usage of technology.
The new environment of audit is a mix of technology (TDA), analytics
(ABA), and human (HBA) efforts just as in the past, but the dramatic
evolution of TDA and ABA makes it necessary to change business
processes, legislation, regulations, and consequently HBA.
The introduction of IT-based analytic monitoring is the introduction of
meta-processes, meta-controls, and meta-management functions. These
meta functions, such as meta-data providing data about data (for
example, in XBRL), meta-control (information about controls being
extracted from ERP systems), or meta-control of controls (information
about the control of controls), provide increasing conceptual confusion
between what auditors and managers should do. The modern IT
environment is aggravating this problem. Migration between functions is
happening and requires new exible conceptualizations.
The need for increased verication due to the many layers of technology
adding opacity and a more complex society has led to many levels of
ALO and the recommendations of the King Commission (Institute of
Directors in Southern Africa, 1994, 2009). The new continuous audit
model aims to liberate from these shackles, creating a new set of
assurance opinions and functions to be provided by the assurance
function in a partnership of management, internal control, internal audit,
and external audit.
6.1 The New CA
The major changes to CA that are emerging and should be permeating
the audit environment, and hopefully standards, are as follows:
r
Progressive adoption of a standard data interface to allow for the
usage of assertion and analytic based apps.
47
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
r
The need to incorporate exploratory data analysis into extant audit
methodology. Liu (2014) proposes such a step in gure 1-11 where
she expects intelligent modules to interface with a wide variety of
data sources.
r
Progressive impounding of audit apps into the operating
environment.
r
The evolution of an audit ecosystem with a progressive level of
automation over nancial and non-nancial systems.
Figure 1-11: EDA and CA (from Liu, 2014)
CA Can Be Redened As
a methodology that enables auditors to provide assurance on a
subject matter for which an entity’s management is responsible,
using a continuous opinion schema issued virtually
simultaneously with, or a short period of time after, the occurrence
of events underlying the subject matter. The continuous audit may
entail predictive modules and may supplement organizational
controls. The continuous audit environment will be progressively
automated with auditors taking progressively higher judgment
functions. The audit will be by analytic, by exception, adaptive,
and cover nancial and non-nancial functions.
48
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
REFERENCES
ACL, New Demands, New Priorities the Evolving Role of Internal Audit: Global Audit
Executives Report (2006).
Alawadhi, A. "The Application of Data Visualization in Auditing." Unpublished
dissertation proposal. (Newark:Rutgers Business School, 2014).
Alles, M. G., A. Kogan, and M. A. Vasarhelyi. "Feasibility and Economics of
Continuous Assurance." AUDITING: A Journal of Practice & Theory 21(1) (2002):
125–138.
—— "Black Box Logging and Tertiary Monitoring of Continuous Assurance
Systems." Information Systems Control Journal 1 (2003): 37–39.
——. "A Relative Cost Framework of Demand for External Assurance of XBRL
Filings." Journal of Information Systems 26(1) (2012): 103–126.
——. "Continuous Monitoring of Business Process Controls: A Pilot
Implementation of a Continuous Auditing System at Siemens." International
Journal of Accounting Information Systems 7.2 (2006): 137–161.
Bailey, A. D., G. L. Duke, G. E. Gerlach, C. E. Ko, R. D. Meservy, and A. B.
Whinston. "TICOM and the Analysis of Internal Controls." The Accounting
Review (April 1985): 186–201.
Bailey, A.D., R. D. Meservy, and P. E. Johnson. "Internal Control Evaluation: A
Computational Model of the Review Process." AUDITING: A Journal of Practice
and Theory (Autumn 1986): 44–74.
Brynjolfsson, E. and A. McAfee. The Second Machine Age: Work, Progress, and
Prosperity in a Time of Brilliant Technologies.KindleEdition.W.W.Norton&
Company, (January 20, 2014): 62–64.
Brown, C. E., J. A. Wong, and A. A. Baldwin. "A Review and Analysis of the
Existing Research Streams in Continuous Auditing." Journal of Emerging
Technologies in Accounting 4 1 (2007): 1–28.
Cash, J. I., A. D. Bailey Jr., and A. B. Whinston. "A Survey of Techniques for
Auditing EDO-Based Accounting Information Systems." The Accounting Review
(October 1977): 813–32.
Canadian Institute of Chartered Accountants/American Institute of Certied
Public Accountants (CICA/AICPA). Continuous Auditing. Research Report.
(Toronto: The Canadian Institute of Chartered Accountants, 1999).
Center for Audit Quality (CAQ). Center for Audit Quality Observations on the
Evolving Role of the Auditor: A Summary of Stakeholder Discussions (2011).
Chiu, V., Q. Liu, and M. A. Vasarhelyi. "The Development and Intellectual
Structure of Continuous Auditing Research." Journal of the Accounting Literature
(2014, forthcoming).
Cushing, B. E. and J. K. Loebbecke. Comparison of Audit Methodologies of Large
Accounting Firms. American Accounting Association, 1986.
Davenport, T. H. Process Innovation: Reengineering Work through Information
Techno l o gy. Boston: Harvard Business School Press, 1992.
Elder, C. A, E. Miyaki, N. Sigolo, and M. A. Vasarhelyi. "Increasing Audit
Efciency Through Continuous Branch KPI Monitoring." Internal Auditor
Magazine. The Institute of Internal Auditors (2013).
Economist. "Special Report on Smart Systems." (November 4, 2010).
49
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Elliott, R. K. "Auditing in the 1990s: Implications for Education and Research."
California Management Review (Summer 1986): 89–97.
Fishman,K.D.The Computer Establishment. New York: McGraw Hill, 1982.
Groomer, S. M. and U. S. Murthy. "Continuous Auditing of Database
Applications: An Embedded Audit Module Approach." Journal of Information
Systems 3 (2) (1989): 53–69.
Halper, F. B., J. P. Snively, and M. A. Vasarhelyi, "CPAS: Knowledge Engineering
and Representation." Paper presented at Second International Symposium on
Expert Systems in Business, Finance, and Accounting, Newport Beach, CA,
November 1989.
——. The Continuous Process Audit System: Knowledge Acquisition and
Representation. Murray Hill: AT&T Bell Laboratories, 1988.
Hammer, M. "Reengineering Work: Don’t Automate, Obliterate!" Harvard Business
Review. (July–August 1990).
Hoogduin, L., K. Yoon, and L. Zhang. "Integrating Different Forms of Data for
Audit Evidence: Markets Research Becoming Relevant to Assurance."
Accounting Horizons (2015, forthcoming).
Information Systems Audit and Control Association. "IT Audit and Assurance
Guidelines" G42, Continuous Assurance (2010).
Institute of Directors in Southern Africa. "King Report on Corporate Governance
for South Africa." (1994).
——. "King Report on Corporate Governance for South Africa." (2009).
Institute of Internal Auditors. Global Technology Audit Guide 3: Continuous
Auditing: Implications for Assurance, Monitoring, and Risk Assessment. Altamonte
Springs: IIA (2005).
Jans, M., M. G. Alles, and M. A. Vasarhelyi. "A Field Study on the Use of Process
Mining of Event Logs as an Analytical Procedure in Auditing." The Accounting
Review vol. 89, no. 5 (September 2014): 1751–1773.
Kaplan, R. S. "Accounting Scholarship that Advances Professional Knowledge
and Practice." The Accounting Review 86 (2011): 367–383.
Kogan, A., M. G. Alles, M. A. Vasarhelyi, and J. Wu. "Design and Evaluation of a
Continuous Data Level Auditing System." AUDITING: A Journal of Practice &
Theory vol. 33, no. 4 (November 2014): 221–245.
Kopetz, H. Internet of Things. Springer: In Real-Time Systems: Springer, 2011,
307–323.
Kozlovski, S. and M. A. Vasarhelyi. "An Audit Ecosystem: A Starting Point with
Denitions, Attributes and Agents." Working paper. Newark: Rutgers Business
School (2014).
Krahel, J.P. and M. A. Vasarhelyi.
19
"AIS as a Facilitator of Accounting Change:
Technology, Practice, and Education." Journal of Information Systems (Fall 2014).
Krahel, J. P "Formalization of Accounting Standards." PhD dissertation. (Newark:
Rutgers University, 2011).
Kuenkaikaew, S. "Predictive Audit Analytics: Evolving to a New Era." PhD
dissertation. (Newark: Rutgers Business School, 2013).
Littley, J. and A. M. Costello. CA/CM as Preventive Care against Fraud. KPMG, 2012.
19
The authors would like to thank Michael Alles, Alex Kogan, and Paul Byrnes for their helpful
suggestions and Qiao Li for her assistance.
50
ESSAY 1: CONTINUOUS AUDITING—A NEW VIEW
Liu, Q. "Exploratory Data Analysis in Auditing." PhD dissertation. (Newark:
Rutgers Business School, 2014).
Loras, T., R. Crossler, J. Long, and B. Trinkle. "Understanding compliance with
BYOD (Bring Your Own Device) Policies Utilizing Protection Motivation
Theory: Bridging the Intention-Behavior Gap." Journal of Information Systems 28
(1) (2014): 209–226.
Mendelson, M., A. Philipovitch, W. Welsh, and R. Zanella. "Securing Cloud-based
Applications." ISACA Journal vol. 1 (2012).
Moftt, K. C. and M. A. Vasarhelyi. "AIS in an Age of Big Data." Journal of
Information Systems vol. 27, no. 2 (Fall 2013): 1–19.
Moharram, B. "Insurance Audit Dashboard." Working paper, Rutgers Business
School, 2014.
O’Leary, D. E. "Reengineering Assembly, Warehouse and Billing Processes, for
Electronic Commerce Using ‘Merge-in-Transit.’" Information Systems Frontiers
vol. 1, no. 4, (2000): 379–387.
Papazoglou, M. P. "Agent-Oriented Technology in Support of E-business."
Communications of the ACM 44(4) (2001): 71–77.
PricewaterhouseCoopers. "Internal Audit Survey: Continuous Audit Gains
Momentum." (2006).
Siegele, L. "Tech Startups: A Cambrian Moment." Economist. (January 18, 2014).
Taleb, N. N. The Impact of the Highly Improbable. Random House, 2010.
Teeter, R.A. "Essays on the Enhanced Audit." PhD dissertation. (Newark: Rutgers
Business School, 2014).
Vasarhelyi, M. A. "Audit Automation: Online Technology and Auditing." The CPA
Journal (April 1985): 10–17.
——. "The CPAS/CCM
20
Experiences: Prospectives for AI/ES Research in
Accounting Information Systems." Paper presented at the ISACA meeting,
Budapest, September 4–7, 1996.
——"A Framework for Audit Automation: Online Technology and the Audit
Process." The Accounting Forum (January 1983).
Vasarhelyi, M. A. and M. G. Alles. "The Galileo Disclosure Model." Version 1.0.
(2006). http://raw.rutgers.edu/Galileo.
Vasarhelyi, M. A., M. G. Alles, and K. T. Williams. "Continuous Assurance for the
Now Economy." A Thought Leadership Paper for the Institute of Chartered
Accountants in Australia, July 2010).
Vasarhelyi, M. A. and F. B. Halper. "The Continuous Audit of Online Systems."
AUDITING: A Journal of Practice and Theory 10.1 (December 1991).
Vasarhelyi, M. A., F. B. Halper, and K. J. Esawa. "The Continuous Process Audit
System: A UNIX Based Auditing Tool." Articial Intelligence in Accounting and
Auditing: Using Expert Systems vol. 2, edited by M. A. Vasarhelyi. (Markus
Wiener Publishers, 1995).
Vasarhelyi, M. A. and A. Kogan. "Big Data in Accounting and Auditing."
Accounting Horizons (forthcoming, 2015).
Vasarhelyi, M. A., A. Kogan, and B. Tuttle. "Big Data in Accounting: An
Overview." Accounting Horizons. In-Press.
20
CPAS stands for Continuous Process Auditing. CCM stands for Continuous Control Monitoring.
51
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Vasarhelyi, M. A., S. Romero, S. Kuenkaikaew, and J. Littley. "Adopting
Continuous Audit/ Continuous Monitoring in Internal Audit." ISACA Journal
vol. 3 (2012).
Vasarhelyi, M. A., J. D. Warren, Jr., R. Teeter, and B. Titera. "Embracing the
Automated Audit." Working paper, Rutgers Business School. (2011).
Vasarhelyi, M. A., and D. C. Yang. "Technological Change and Management
Information Systems," Proceedings of the Twenty-First Annual Hawaii
International Conference on System Sciences (1988): 191–197.
Wei, J. "How Wearables Intersect With the Cloud and the Internet of Things:
Considerations for the Developers of Wearables." Consumer Electronics
Magazine 3 (3). Institute of Electrical and Electronics Engineers. (2014): 53–56.
Zhang, L., Pawlicki, A., McQuilken, D., and Titera, W. "The AICPA Assurance
Services Executive Committee Emerging Assurance Technologies Task Force:
The Audit Data Standards (ADS) Initiative," Journal of Information Systems
(Spring 2012).
52
ESSAY 2
The Current State of
Continuous Auditing
and Continuous
Monitoring
1
Paul Eric Byrnes, CMA
Brad Ames, CPA, CISA, CRMA
Miklos Vasarhelyi, PhD
INTRODUCTION
The AICPA Assurance Services Executive Committee’s Emerging
Assurance Technologies Task Force is drafting a series of white papers to
serve as an update to the 1999 "Red Book" project entitled Continuous
Auditing (CICA/AICPA 1999). The primary purpose of this introductory
white paper is to facilitate an understanding of the extent to which the
applications of CA/CM have changed during the previous 12-year
period. In addition, this paper is designed to briey explore related issues
and set the stage for associated white papers that will be subsequently
developed.
Data for this undertaking was collected using a two-phase approach.
First, a comprehensive, open-ended questionnaire was formulated and
distributed primarily to the Big 4 accounting rms in an effort to assess
variables pertinent to CA/CM usage and perspectives. Second, follow-up
interviews were conducted both to clarify survey responses as well as to
1
First published in October 2012 from the AICPA Assurance Services Executive Committee
(ASEC) Emerging Assurance Technologies Task Force.
53
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
obtain complementary information. As the project unfolded, four
primary questions emerged as the most relevant in addressing
established objectives:
1. What are rms doing with CA/CM?
2. What CA/CM products and services could rms provide?
3. What could organizations such as the AICPA or Public Company
Auditing Oversight Board (PCAOB) do to facilitate the adoption
and use of CA/CM?
4. Do rms have the necessary internal expertise to provide the array
of CA/CM options and what is the desired skillset for dispensing
such services?
As data was accumulated and analyzed, some trends were observed that
ultimately provided a snapshot of where CA/CM exists today. There are
major challenges and barriers to achieving widespread adoption and
proliferation of CA/CM practices. Interestingly, this is particularly
evident in the area of external auditing, although there has been some
limited progress in the internal audit area (Vasarhelyi, et al 2012). In
presenting the relevant ndings, each of the preceding primary questions
will be addressed in order of original presentation.
CURRENT ENVIRONMENT
The general view is that not much is currently being done with CA/CM.
In reality, this synopsis is not as pessimistic as it appears. Specically,
some positive gains in usage are being noted in the area of internal
auditing. For the most part, this entails the collaboration of advisory
services divisions of public accounting rms with internal audit clients in
implementing CA/CM devices and methods. There are also instances
where organizations are outsourcing their internal audit functions and
this could conceivably create additional CA/CM opportunities for audit
rms. On the other hand, direct CA/CM implementations by external
auditors have not noticeably increased and there are specic reasons for
this situation. One important consideration is the idea that CA/CM is a
costly undertaking and payback period, which is often projected to be
quite lengthy. The perceived instability of audit relationships is an
element that interacts unfavorably with the cost and payback period
variables associated with potential projects. Another impediment exists
because many businesses are protective of their data and, therefore,
reluctant or unwilling to allow comprehensive and ongoing access to
systems by outside parties, including external auditors.
On a positive note, the public accounting arena seems to be encouraging
organizations to internally develop and implement CA/CM programs. In
54
ESSAY 2: THE CURRENT STATE OF CONTINUOUS AUDITING AND CONTINUOUS MONITORING
doing so, auditors are ultimately seeking to leverage the use of these
devices in conducting external audits. For example, if an audit client has
an effective CA/CM system in place, then the external audit team may be
positioned to use data generated by this system in order to conduct the
external audit in a more efcient and effective manner. In such cases, this
has the potential to create benecial situations for both parties. In other
words, the auditing rm would have access to more comprehensive and
timely information and would therefore be able to perform a higher
quality audit that consumes fewer resources relative to traditional
methods. In addition, because of increased efciency, the audited
organization would presumably sacrice fewer resources in obtaining the
external audit services. Clearly as more businesses recognize the value of
CA/CM, one might expect to see an increase in this type of behavior,
particularly if favorable outcomes are likely to exist for the involved
parties. To facilitate an understanding of this collaboration process, the
appendix of this white paper presents a recent implementation at
Hewlett-Packard Company. At this point, with an understanding of the
general state of CA/CM, an exploration of potential CA/CM products
and services will now be investigated to provide an appreciation of the
manners in which they could affect both the accounting profession and
business community.
PRODUCTS AND SERVICES
Initially, CA/CM tool usage might be envisioned as existing on a
continuum, from relatively basic monitoring of a particular target area of
risk such as accounts payable, to very elaborate auditing systems that
yield continuous assurance capabilities such that audit opinions can be
maintained in an ongoing manner. Specic CA/CM service opportunities
include converting from manual to automated data, controls, and
processes; designing controls around processes; formulating tests and
monitoring routines; dispensing operational risk management services;
and providing full service packages including tools, installation, setup,
training, and maintenance.
CA/CM consists of many diverse elements and may be implemented at
various levels of sophistication. One of the key features of CA/CM is its
ability to provide relevant information in more of a real time context. If a
solution is installed, maintained, and utilized as intended, it has the
capability to assist in mitigating or even preventing problems in
identied risk areas. This is in sharp contrast to the reactionary context in
traditional external auditing, whereby annual sampling and testing is
conducted to discover whether problems occurred during the scal
period under investigation.
55
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
The traditional approach is suboptimal for at least two reasons. First, the
manual audit is based on a sampling of records in identied risk areas
and therefore may fail to capture all relevant data. Second, even if
problems are uncovered, the lag between event occurrence and detection
may be too signicant to allow for sufcient remediation and recovery.
For example, Company X has an annual audit conducted in January
relative to the previous calendar year. An examination of sampled
transactions uncovered a material fraud that was successfully
perpetrated during the rst quarter of the accounting period in question
and resulted in a signicant diversion of assets. In this case, the damage
went completely undetected for an extended period and, as such, the
likelihood of the rm fully recovering from the loss is lower than if the
issue had been identied sooner. An effective CA/CM mechanism would
have uncovered the fraudulent activity in the formative stages and, as a
consequence, resource loss could have been minimized or perhaps
avoided entirely.
CA/CM has the potential to radically reformulate the manner in which
businesses operate. The area shows real promise in contributing to
organizational efciency, effectiveness, and long-term protability.
However, areas of CA/CM appear to be struggling for acceptance and
are in a state of tenuous growth. Moving forward, perhaps more
advocacy mechanisms are necessary.
PROMOTION EFFORTS
During discussions with survey participants, three items in particular
emerged as potential keys for achieving increased implementation of
CA/CM practices. First, standards modication by the PCAOB was cited
as being important in facilitating a shift away from the old workplace
mentality of manual sampling and testing in audits to an automated and
comprehensive approach with CA/CM as the foundation. For example,
during the external audit, certain actions, such as physical observation of
inventories, are required regardless of the robustness of controls in place
throughout the organization. The argument against such observation
activities holds that, if controls are sufcient, particular verication
routines become unnecessary and wasteful of resources. Generally, some
believe that auditing standards have not been rened appropriately as
changes in technology, processes, and controls have evolved.
Consequently, many of the current auditing standards are viewed as
antiquated and irrelevant.
Second, many concerns about CA/CM relate to its apparent level of
sophistication. As such, another suggestion is that the AICPA could assist
with advocacy efforts by periodically issuing guidance or white papers
on various topics of relevance. More specically, such subjects could
56
ESSAY 2: THE CURRENT STATE OF CONTINUOUS AUDITING AND CONTINUOUS MONITORING
include explaining CA/CM in terminology that facilitates enhanced
understanding, training relative to implementation and usage,
demonstrating application value, and specifying how CA/CM might
ultimately transform business operations. Basically, if practitioners are
made more aware of CA/CM, understand its concepts and applications,
and identify with the overall value proposition, these individuals will be
more likely to embrace and pursue CA/CM initiatives. This white paper
and the series of ensuing AICPA CA/CM white papers are intended to
start addressing this opportunity to educate the market concerning the
potential of CA/CM as well as identify with the inclusion of audit data
standards within this domain.
Third, it is believed that a formal endorsement of the desirability of
CA/CM by standard setting bodies such as the PCAOB would be useful
for transitioning organizations from traditional manual auditing
methodologies to the more automated domain. However, until a greater
level of awareness is achieved, it is probable that CA/CM will continue
to struggle for acceptance. Whether or not signicant momentum is
eventually generated, discussions logically transition to skillset
considerations and whether accounting practitioners possess the
capabilities necessary to handle CA/CM initiatives.
SKILLS REQUIRED
Preliminary indications are somewhat mixed concerning whether current
accounting professionals maintain the competencies to perform CA/CM
services. One view is that the present generation of accountants lacks the
requisite skillset to sufciently provide these services. Another
perspective contends that the desired expertise is scattered throughout
the rm and, as a result, there is no single individual who would
typically be regarded as a CA/CM expert. These viewpoints may initially
appear somewhat discouraging. However, even if these perceptions are
able to be generalized, they still do not pose an insurmountable threat,
particularly when one reects upon the desired skillset elements as
dened by accounting rms in this study. The competencies identied by
survey participants include the following:
r
An audit foundation
r
Knowledge of business processes, controls, and inherent risks
r
Internal audit experience
r
Familiarity with audit planning, audit processes, and forensic
accounting
r
An understanding of data extraction tools (IDEA, ACL)
r
Data analytics background (regression, ANOVA, data mining,
SQL, probabilities)
57
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
r
Knowledge in statistics
r
Technical skills (ERP, programming)
r
Professional skepticism and judgment
The areas delineated in the preceding list certainly present challenges for
many current accounting students and practitioners. Some of the
elements are not substantive components of traditional four year
accounting programs (for example, technical and analytical areas) and
certain items, such as forensic accounting, reect relatively recent
additions to the accounting discipline. Some capabilities would also be
primarily developed through relevant and extensive eld experience.
Consequently, a CA/CM specialist would most likely be a well-seasoned
practitioner who has developed extensive audit experience, pertinent
technical and analytical expertise, and the ability to employ professional
skepticism and judgment as necessary. Such an individual would also
need to engage in ongoing education relative to emerging concepts,
trends, and technologies in auditing and accounting. Therefore, the
combination of diverse attributes needed to be considered a CA/CM
expert is such that this designation might be expected to be held by a
small minority. Whatever the case, it may actually be more realistic to
envision that many CA/CM initiatives of the near future would be
handled by cross-functional teams, especially when the scope of activity
is substantial. In such cases, perhaps a CA/CM specialist with proven
project leader experience could be enlisted to guide the team in
completing all necessary objectives.
SUPPLEMENTAL FINDINGS
Respondents also communicated other points of interest concerning
CA/CM. The results were generally mixed and suggested that additional
efforts are needed relative to the promotion and adoption of CA/CM
practices. More specically, participants provided revealing commentary
in areas such as recruitment tactics, client perceptions, and prioritization
issues.
Only half of the rms indicated they actively recruit for CA/CM. It is,
however, generally larger rms that emphasize hiring individuals with
CA/CM backgrounds and skills. Furthermore, audit automation is
currently in a state of development and is slowly increasing in terms of
application. Taken collectively, this might indicate that smaller rms will
eventually be more inclined to recruit for CA/CM as adoption and
utilization continue to expand.
Some respondents felt that client understanding of CA/CM was quite
limited, particularly at the top management or board levels. This is a
disconcerting situation for several reasons. Initially, it is well documented
58
ESSAY 2: THE CURRENT STATE OF CONTINUOUS AUDITING AND CONTINUOUS MONITORING
that tone-at-the-top is a dimension that has a signicant inuence upon
organizational culture and resulting perspectives and behaviors
(Merchant and Van der Stede 2007). Furthermore, this tone can be a key
factor in determining whether or not certain initiatives are promoted,
valued, and explored at lower management and operational levels. If top
management maintains an insufcient understanding of CA/CM
products and services, there is a reduced likelihood that employees at
lower levels of the organization will be poised to actively pursue such
solutions. Therefore, if CA/CM is to move forward more uidly, it is vital
that personnel at the board and top management levels better understand
and identify with the benets of CA/CM products and services.
Participants clearly stipulated that a high priority exists relative to
improving the use of CA/CM in conducting nancial statement audits.
Furthermore, it is believed that the accounting profession needs to
establish a vision for the future concerning the CA/CM domain. Related
to this vision concept, respondents provided three important guidelines.
First, they mentioned that audit processes need to be modied as changes
occur with respect to technology and information availability. Second,
they indicated that a greater utilization of CA/CM products and services
is essential. Third, respondents argued that robust CA/CM is ultimately
desired such that audit opinions may be available on a continuous basis
and removed when substantive negative evidence surfaces. This nal
viewpoint has been alternatively referred to as the evergreen opinion and
might well reect the optimal state concerning application of CA/CM in
practice.
CONCLUSIONS
In summary, organizations are not yet reaping the entire array of benets
that CA/CM could yield. Although some noteworthy gains have been
made in internal auditing, there has not been a corresponding increase in
external audit applications. In addition, there is an extensive set of
products and services that may be provided by practitioners under the
CA/CM umbrella. However, these offerings require a diversied skillset
for effective implementation and management. At this juncture, there is a
level of incongruence between the competencies needed and the skills
being acquired by the typical modern accounting professional. CA/CM
has also not yet established a fully successful marketing campaign. In
response, if the value of this approach can be effectively demonstrated,
documented, and disseminated, and if groups such as the AICPA and
PCAOB become key players in the education and advocacy efforts, it is
plausible to envision that CA/CM will eventually realize its full potential.
59
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
REFERENCES
Alles, M., Brennan, G., Kogan, A., and Vasarhelyi, M. A., 2006. "Continuous
monitoring of business process controls: A pilot implementation of a
continuous auditing system at Siemens," International Journal of Accounting
Information Systems 7 (2): 137–161.
CICA/AICPA, 1999. Research Report: Continuous Auditing. Toronto, Canada: The
Canadian Institute of Chartered Accountants, American Institute of Certied
Public Accountants.
Elliott, R. K., 1994. "The future of audits," Journal of Accountancy 178 (3): 74–82.
Elliott, R. K., 2002. "Twenty-rst century assurance," Auditing 21 (1): 139.
IIA, 2005. Global Technology Audit Guide (GTAG) 3: Continuous Auditing:
Implications for Assurance, Monitoring, and Risk Assessment.
ISACA, 2010. IT Audit and Assurance Guidelines: G42 Continuous Assurance.
KPMG, 2009. What Is Driving Continuous Auditing & Continuous Monitoring Today?
White Paper. KPMG LLP.
Merchant, K. A., and Van der Stede, W. A., 2007. Management Control Systems:
Performance Measurement, Evaluation and Incentives, 2nd ed. Harlow, England:
Prentice Hall.
Vasarhelyi, Miklos A., and Fern B. Halper, 1991. "The Continuous Audit of Online
Systems," Auditing: A Journal of Practice and Theory 10 (1): 110–125.
Vasarhelyi, M., Romero, S., Kuenkaikaew, S., and Littley, J., 2012. "Adopting
Continuous Audit/Continuous Monitoring in Internal Audit." ISACA Journal 3.
APPENDIX—CONTINUOUS AUDITING AND
CONTINUOUS MONITORING IN ACTION
Introduction
An excellent example of how rms are leveraging the use of technology
in conducting monitoring and auditing activities may be noted at
Hewlett-Packard Company (HP). In this context, HP’s internal auditing
department has been an instrumental force in devising an automated
system to capture, analyze, and communicate key business data, metrics,
and transactions to support better decisions regarding risk. Furthermore,
HP demonstrates an ongoing commitment to embracing current
technologies in conducting business analysis and oversight, giving the
organization a denite competitive advantage. One recent
implementation example is the Decision Support and Analysis Service
Internal Audit (DSAS/IA) Database Project, which comprises several
important features. At the onset, it should be noted that all data
presented in subsequent gures is completely ctitious and designed
only for illustrative purposes.
60
ESSAY 2: THE CURRENT STATE OF CONTINUOUS AUDITING AND CONTINUOUS MONITORING
SAP Key Performance Indicator
Initially, utilizing the SAP environment, the SAP key performance
indicator (KPI) solution has been deployed to capture and communicate
pertinent KPI measures as needed. This information is then made
immediately available to the internal audit staff via Web download and
retained in the DSAS/IA Oracle database for future access and usage.
This clearly provides for the dissemination of critical data analytics to
users such that they are readily positioned to promptly respond to
changing business conditions and circumstances.
DSAS/Audit Command Language
SAS/Audit Command Language (ACL) is used for extracting relevant
transactional data and les from the DSAS/IA database, as well as
accumulating facts from other points such as unstructured data from
external feeds. In addition, when an instance of data extraction occurs,
the material is presented through a website. From this location, the
information is able to be transferred to and accessed by authorized
internal audit staff, external audit personnel, and business unit
management to assist with oversight or operational activities.
Furthermore, the data provided to external audit is prevalidated by the
internal audit staff and, therefore, may be readily relied upon. This is an
extremely important feature that builds trust in the data and facilitates
audit effectiveness and efciency.
DSAS Database
The DSAS Database Table Content Query Screen enhances query
development and processing via a user-friendly point-and-click
atmosphere containing convenient drop-down lists, selection windows,
selection bubbles, and so forth. In addition, query information is
presented in the Standard XML Excel Spreadsheet for Reports
environment from which the material may be exported to, analyzed, and
evidenced in the Microsoft Excel Spreadsheet context according to an
audit data standard. Furthermore, this query building mechanism is
often used in providing information to external auditors. These auditors
are then able to perform an array of relevant operations with the
information, such as benchmarking activities. Overall, the
aforementioned functionality greatly simplies query generation and
empowers users to better harness the capabilities that query building
offers (gures 2-1 and 2-2).
61
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 2-1
The Decision Support and Analysis Service (DSAS) Database Table
Content Query window assists users in the query development
process with various prompts, selection options, and drop-down
lists. In addition, it allows for providing information in accordance
with audit data standards to external auditors.
Figure 2-2
According to audit data standards, the following is a query result
set as a function of values and options chosen in the Decision
Support and Analysis Service (DSAS) Database Table Content
Query Window.
62
ESSAY 2: THE CURRENT STATE OF CONTINUOUS AUDITING AND CONTINUOUS MONITORING
Dashboard Feature
The system contains an internal audit dashboard feature whereby key
metrics and trends are captured and displayed in a variety of graphical
and tabular formats that enable users to easily visualize pertinent
information and detect problems at a glance. The main internal audit
dashboard window also maintains other functionalities including, but
not limited to, executive summaries (gure 2-3).
Figure 2-3
The Internal Audit Dashboard window displays pertinent metrics
in a convenient fashion.
The internal audit (IA) dashboard also provides critical operational
information, group account data and ratios, compliance and performance
metrics, business reporting analyses, and so forth. First, the auditors
believe that manual journal entries (MJEs) applied during the standard
closing process carry more risk than other MJEs. With this in mind, the
Workday Analysis feature depicts MJE activity occurring during the
accounting closing period and is able to display this information for three
accounting periods at a glance. In general, MJEs generated during the
rst day of closing (WD1) represent less risk than those posted on
subsequent days. Furthermore, the risk and day attributes maintain a
positive relationship such that risk increases as MJEs are applied later in
the closing process. As such, periods that have a greater percentage of
MJEs applied near the conclusion of closing may be targeted for further
investigations. For added utility, the audit tool displays MJE activity by
account category to isolate areas where deeper inspections are warranted
(gure 2-4).
63
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 2-4
Financial Close Workday Analysis: In the upper charts, closing
period manual journal entry (MJE) activity is displayed for each
accounting period by workday (WD) using a color coding scheme.
In the lower table, MJE metrics are presented by account category.
In this example, the account most impacted by post-closing MJEs
in each period is Cash. This nding, in addition to the fact that
Cash is inherently a high-risk account, suggests that further
analyses may be advised in this area.
(Amounts in Billions)
Second, the general ledger (GL) Account Wise Analysis function
aggregates information by account type to display various pertinent
metrics. In fact, multiple levels of aggregation are available to facilitate
drilldown operations. For example, a level 2 view, which is a single
drilldown from level 1, results in GL account groupings identical to those
shown in gure 2-5. Similar to other IA dashboard tools, emphasis is
again placed upon looking at MJE activity. However, additional
information is captured relative to trial balance (TB) and manual journal
voucher (MJV) values.
64
ESSAY 2: THE CURRENT STATE OF CONTINUOUS AUDITING AND CONTINUOUS MONITORING
Figure 2-5
The GL Account Wise Analysis screen depicts pertinent trial
balance (TB), manual journal voucher (MJV), and manual journal
entry (MJE) information by account type. As an example, MJE
debits in the Cash account group represent 75 percent of the TB
debit amount (MJV Debit amount or TB Debit amount). In this
example, the MJE information in the Cash and Equities groups is
highlighted for further investigation.
(Amounts in Billions)
Third, the KPI tool provides specic risk measures, displaying each in
terms of percentage of total MJEs and aggregate number of postings.
Furthermore, for each attribute, the system divulges the 5 countries that
contribute most signicantly to the associated KPI value. For example, in
gure 2-6 there were 318 MJEs posted by terminated users during the
period in question. Also, the 5 countries contributing most to the
accumulation of these entries were 34, 59, 213, 106, and 101. Given the
high risk associated with this KPI, the entire array of transactions might
warrant further analysis. In addition, all or a subset of the 5 identied
countries might be subjected to more indepth outlier analysis,
particularly those countries demonstrating a pattern of riskiness via
other analytical procedures. Country 34, for example, is also reported
more frequently in other KPIs and this suggests that this region deserves
a closer examination. Taken collectively, the KPI feature contributes to
making sampling decisions relative to detailed testing operations as well
as identifying locations having the highest risk factors.
65
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 2-6
The Key Performance Indicator (KPI) Performance tool supplies
useful metrics for various risk assessments and assists with the
coordination of detailed testing routines.
Fourth, the Business Area Analysis functionality isolates pertinent MJE
debit and credit values by business segment and amount range and this
contributes to optimizing efciency relative to auditing processes. For
example, for the accounting period depicted in gure 2-7, business
segment F generated $955 million in MJE debit postings wherein each
entry was less than $10,000. Whether this was problematic would depend
upon a number of factors, such as recent acquisitions or large accruals.
Whatever the case, having the capability to assess MJE activity via the
dimensions of business unit and amount grouping offers yet another
valuable view when conducting audit activities.
66
ESSAY 2: THE CURRENT STATE OF CONTINUOUS AUDITING AND CONTINUOUS MONITORING
Figure 2-7
The Business Area Analysis window presents manual journal
entry (MJE) debit and credit information by amount range at the
business unit level.
Last, the Grid Analysis feature classies countries and regions according
to total TB amounts as well as the percentage of total MJVs. More
specically, in gure 2-8 the 5 categories delineated down the left side of
the matrix correspond to the percentage of total journal vouchers that are
manually created. Similarly, the 4 groups displayed across the top of the
grid represent total trial balance dollar values. Essentially, each country
or region is ranked along these 2 dimensions such that riskiness may be
readily observed. In gure 2-8, Country_55 exclusively occupies the
block representing the intersection of row 1 and column 1 (category of
least risk). For the period under analysis, this country has less than $.006
billion in total TB value, generated no more than 56 percent of its journal
vouchers manually, and would be perceived as the least risky country via
the Grid Analysis operation. In addition to evaluating riskiness, this
procedure also facilitates the discovery of better performing countries
and regions such that best practices may be identied and disseminated.
67
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 2-8
The Grid Analysis function categorizes countries and regions by
total trial balance amounts and manual journal vouchers as a
percentage of total journal vouchers.
In addition to the preceding capabilities, the continuous auditing and
continuous monitoring system is able to perform statistical analyses and
employ methods such as Benford’s Law and linear regression. For
example, related data of interest may be used to construct a line of best t
via linear regression. A condence band may then be established for this
line and resulting outliers identied. Any points falling outside and
above the condence interval would be regarded as problems requiring
closer investigation. Conversely, any points located within or below the
condence band would not be perceived as troublesome. However, such
points would likely be explored in an effort to discover and communicate
pertinent information relative to best practices. In a specic example
illustrated in gure 2-9, a positive relationship has been noted by country
between trial balance amounts and number of manual journal entries. As
such, a regression equation was constructed that reliably represents this
association. Then, a 95 percent condence interval was created around
the regression line. When the line, condence band, and actual data
points were plotted, issues became glaringly apparent.
68
ESSAY 2: THE CURRENT STATE OF CONTINUOUS AUDITING AND CONTINUOUS MONITORING
Figure 2-9
The following gure represents use of linear regression in
conducting data analysis. The three points identied outside and
above the condence band pertain to unfavorable outliers that
need further investigation (Countries 1, 2, and 3). The two points
falling outside and below the condence band pertain to favorable
outliers that would likely be investigated to uncover company
best practices information.
The ability of the preceding regression procedure to assist in the
discovery of outliers and exceptions clearly makes the oversight process
signicantly more efcient.
In addition to the previously noted internal audit dashboard benets, the
module offers other noteworthy strengths:
1. Any of the pertinent data may be readily provided to external
audit personnel and other interested parties through an intranet
website.
2. Extensive drilldown capabilities facilitate data disaggregation
tasks. For example, in conducting global analytics, a user may
start with region (for example, the Americas) and proceed to drill
down through the layers of geography, business unit, process, and
so forth (gure 2-10). Ultimately, the individual could isolate a
particular record of interest to obtain detailed information, such as
the name of the employee who posted or approved the transaction.
69
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 2-10
The global analytics Monthly Analysis tool contains drilldown
functionality for data disaggregation tasks. In the following gure,
aggregated monthly accounting period information is displayed
that pertains to trial balance amounts and manual journal voucher
values and percentages.
As with query reporting, functionality is present in many situations to
export information to the Microsoft Excel Spreadsheet environment for
further analysis and modeling.
Clearly, the IA dashboard signicantly simplies the management,
monitoring, and oversight functions and produces a vast array of useful,
reliable, and timely information for decision making purposes.
In summary, the DSAS/IA Database Project is focused on the leveraging
of current technologies to automate organizational management,
monitoring, and auditing. By accumulating key metrics, statistics, and
other relevant information in an ongoing manner, the system positions
users to respond to changing business circumstances as associated events
and transactions occur. Furthermore, the auditor is optimally positioned
to detect issues promptly as they occur, which may ultimately serve a
predictive or preemptive purpose. At this point, the auditor is poised to
ensure that its business units are appropriately deploying the rm’s
scarce resources and, thus, assisting in optimizing protability and prot
growth moving forward.
70
ESSAY 3
Evolution of Auditing:
From the Traditional
Approach to the Future
Audit
1
Paul Eric Byrnes, CMA
Abdullah Al-Awadhi, PhD
Benita Gullvist, DSc
Helen Brown-Liburd, PhD, CPA
Ryan Teeter, PhD
J. Donald Warren, Jr., PhD
Miklos Vasarhelyi, PhD
INTRODUCTION
Auditing is currently at a critical juncture. Specically, advances in
information technology in conjunction with real-time approaches to
conducting business are challenging the auditing profession. As such, the
primary purpose of this essay is to examine the extent to which the
auditing discipline in the United States has advanced and identify the
trajectory it might take if it is to continue to thrive and provide long-run
value to society at large.
1
First published 2012. From the AICPA Assurance Services Executive Committee (ASEC) Emerg-
ing Assurance Technologies Task Force.
71
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
ABRIEF HISTORY OF AUDITING IN THE
UNITED STATES
Although auditing procedures have been relied upon for many years, the
formal practice of auditing has been in existence for a relatively short
period. In addition, emphasis has historically been placed on a periodic,
backward-looking approach whereby key events and activities are often
identied long after their occurrence or simply undetected. Given that
recent developments and technologies facilitated a movement away from
the historical paradigm and toward a more proactive approach, it is
essential that auditors understand what the future audit entails and how
they might begin to envision a logical progression to such a state. To
enhance this comprehension, it is advisable to consider how auditing has
evolved from its formal beginnings in the early twentieth century.
The Industrial Revolution and the resulting explosion in growth of
business activity led to widespread adoption of auditing methods. The
railroads, in their efforts to report and control costs, production, and
operating ratios, were major catalysts in the development of the
accounting profession within the United States (Chandler 1977).
Specically, rms became aware of the need for mechanisms of fraud
detection and nancial accountability, and investors increasingly relied
upon nancial reports as corporations began to participate in the stock
market. Although these issues prompted an expansion in the use of
accounting and auditing mechanisms, it was after the stock market crash
of 1929 that auditing became an obligatory process in the United States.
In particular, the Securities and Exchange Act of 1934 created the
Securities and Exchange Commission (SEC). Among other
responsibilities, the SEC was initially given authority for the
promulgation of accounting standards as well as auditor oversight
functions. In addition, the SEC was required to enforce the mandate that
publicly traded U.S. companies submit various periodic reports to the
agency in a timely fashion. To assist the SEC with ensuring that these
reports were created in accordance with generally accepted accounting
principles (GAAP), public accounting rms were eventually required to
provide certain assurances about the information.
Many of the audit practices existing during the period that immediately
followed were not conducted independently and, instead, simply relied
upon information from management personnel. Furthermore,
renements of audit standards generally consisted of reactionary
measures that occurred in response to signicant negative business
events. For example, audit tasks such as physical inspection of
inventories and conrmation of receivables were optional until
fraudulent activities were uncovered at McKesson & Robbins in 1939. As
72
ESSAY 3: EVOLUTION OF AUDITING
a result, the AICPA issued Statement on Auditing Procedure (SAP) No. 1
in October 1939 and it required that auditors inspect inventories and
conrm receivables. Consequently, auditors became responsible for
auditing the business entity itself rather than simply relying upon
management verication routines.
Following this, auditing by inspection and observation became the norm.
Even as automated accounting systems began to appear in the 1950s,
manual auditing procedures continued to be used exclusively. For
example, in 1954, UNIVAC was unveiled as one of the rst operational
electronic accounting systems in the United States. However, auditors
only began to seriously consider auditing in the computerized context in
the early 1960s; two specic events prompted this transition.
First, in 1961 Felix Kaufman wrote Electronic Data Processing and Auditing.
The book compares auditing around and through the computer.
Historically, auditing "around the computer" entails traditional manual
procedures in which the existence of automated equipment is ignored. As
such, the computer is treated as a black box. In this context, auditors rely
upon physical inputs to and outputs from automated devices and do not
concern themselves with how processing actually occurs within the
system(s). Conversely, auditing "through the computer" involves actual
use of computer systems in testing both controls and transactions. Finally,
auditing "with the computer" entails direct evaluation of computer
software, hardware, and processes. Consequently, auditing through the
computer or with the computer is able to provide a much higher level of
assurance when contrasted with auditing around the computer.
Second, International Business Machines (IBM) released its IBM 360 in
1963 and this device made computing more affordable than ever. Clearly,
these developments collectively signaled a paradigm shift in terms of
how accounting activities were to be conducted in the future and
facilitated serious consideration of movement away from the traditional
manual audit.
Notwithstanding the progression toward computerized accounting,
many auditors continued to audit around the computer and the minority
who elected to audit through the computer relied on an array of
proprietary programs that were expensive, cumbersome, inefcient, and
in need of constant reprogramming. For example, Cangemi and Singleton
(2003) mention that in 1967, one rm developed between 150 and 250
unique auditing programs. Furthermore, nearly 80 percent of these
programs required signicant code modication in the subsequent year
because of computer system enhancements and changes in audit
requirements. The introduction of AUDITAPE by Haskins & Sells in 1967,
a card oriented auditor-friendly computer assisted audit tool (CAAT),
encouraged additional auditors to consider moving into the automated
73
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
domain. In particular, AUDITAPE allowed nontechnical auditors the
increased ability to audit through the computer and facilitated the
creation of several general auditing software (GAS) programs from 1968
through the late 1970s. In conjunction with the development of these
initial audit programs, Davis (1968) alerted auditors to the idea that they
would simply not be able to ignore electronic data processing (EDP) in
accounting systems when performing audits. In addition, he explained
how and when auditing around the computer might be accomplished,
but advised that an evaluation of internal controls as both a review and
test of system reliability (audit of the computer) would still need to be
performed. Davis had a signicant and positive effect on the evolution of
audit theory and practice. Moving forward, the 1970s saw 2 major
developments that dramatically altered the accounting and auditing
landscapes.
First, the Equity Funding Corporation scandal of 1973 is sometimes
perceived as the single most signicant event in EDP audit history. In
particular, the organization committed acts of fraud between 1964 and
1973 (Seidler et al. 1977). Essentially, managers created false insurance
policies and commission income to articially inate prots and stock
price and used a variety of mechanisms to conceal the activities. For
example, when auditors attempted to conrm receivables via phone calls
to customers, switchboard operators at Equity Funding would simply
connect the calls to employees who would subsequently conrm the
balance information. When the fraud was eventually unearthed in 1973,
Equity Funding had $2 billion in phony insurance policies and this
reected roughly 67 percent of the total balance in that general ledger
account. In reection, it was determined that an EDP audit would
uncover the fraud much sooner. This determination was made primarily
because all of the false policies were posted to department number 99,
whereas legitimate policies were not applied there.
Whatever the case, the Equity Funding debacle was instrumental in
mandating a shift from auditing around the computer. Furthermore, the
incident prompted the review of existing audit processes in an effort to
address internal controls and audit procedures for information systems.
As a consequence, large accounting rms, previously known as the Big 8,
established units consisting of EDP specialists to audit information
systems. Smaller accounting rms often maintained contracts with
information systems professionals to assist in auditing such systems.
Second, the Foreign Corrupt Practices Act (FCPA) of 1977 had substantial
implications for accountants. Basically, the FCPA prohibited American
companies from bribing foreign ofcials to obtain business and required
these rms to have mechanisms in place to detect such activities. In
addition, the FCPA required companies registered with the SEC to
maintain their books and records such that transactions were accurately
74
ESSAY 3: EVOLUTION OF AUDITING
and fairly reported and consistently employ adequate systems of internal
controls. Consequently, U.S. companies were forced to implement
signicantly more robust accounting systems as well as internal controls
within those systems.
During the next 25 years, many of the noteworthy events involving
auditing of information systems pertained to the development and
renement of automated vendor offerings designed to increase
effectiveness and efciency in auditing. The advancement and
proliferation of technologies, such as the personal computer, led to
electronic data processing becoming more widespread within
organizations (Davis 1968). As an example, the author shows that the
number of computers installed in U.S. based companies increased
fourfold between 1962 and 1967. Along with this extensive distribution of
computing power and security risk came the increasing demand and
need for micro-based computer assisted audit tools (CAATS) designed to
aid in automating the audit process. In fact, the exibility and power of
CAATS helped to bring improved audit quality and speed when dealing
with the increase in data availability associated with automated systems.
In response to the expanding demand for CAATS, vendor-based
solutions began to appear in the marketplace and the need for accounting
rms to continue developing proprietary in-house audit tools was greatly
diminished. For example, standardized audit tools such as Audit
Command Language (ACL) and Interactive Data Extraction and Analysis
(IDEA) emerged and offered signicant advantages over the
COBOL-based programs of the previous period. Moving forward, such
tools are periodically rened and continue to provide valuable assistance
to those seeking to audit through the computer today. Although CAATS
have been instrumental in encouraging a shift away from traditional
manual auditing, another fairly recent development has also had a
signicant effect.
Specically, passage of the Sarbanes-Oxley Act (SOX) in 2002 imposed
sweeping changes on publicly traded companies and the accounting
profession. SOX established that assurances about internal control
practices and operations as well as nancial reporting quality were the
responsibility of both management and auditors. Furthermore, SOX
caused the accounting discipline to devote more attention to addressing
fraud during the course of an audit. For example, Statement on Auditing
Standards (SAS) No. 99, Consideration of Fraud in a Financial Statement
Audit (AICPA, Professional Standards, AU-C sec. 240), requires auditors to
design audit procedures that provide reasonable assurance of detecting
fraud that could have a material effect on the nancial statements.
As is evident from the preceding discussion, auditing maintains a very
interesting past and renements have occurred progressively along the
75
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
way that ultimately established capabilities for an improved audit
experience. However, barriers continue to exist in evolving toward the
future audit. For example, the traditional auditing paradigm whereby
transactions are sampled based upon risk considerations continues to be
prevalent in the auditing profession today. Unfortunately, this process
often fails to maximize utility in the information age. Conversely, the
future audit that relies upon the leveraging of technologies and processes
has the capability to expand analyses of a rm’s operating activities and
thus provide improved audit quality. As an example, Kuhn and Sutton
(2006) examined fraudulent capital expenditures at WorldCom and
determined that, where the manual auditing system failed, a properly
structured continuous assurance (CA) system would successfully detect
suspicious transactions in a timely fashion. Perhaps with effective CA
systems in place, the WorldCom disaster and others like it could have
been avoided entirely.
In further support of the future audit, it is estimated that total global
fraud losses were more than $2.9 trillion in 2009 (Association of Certied
Fraud Examiners 2010). More important, this gure continues to rise.
Although some aspects of the traditional audit will continue to hold
value, the audit of the future provides opportunities to increase the use of
automated tools and remains a key for offering improved assurances
relative to the responsible management and utilization of stakeholder
assets. Moving on, with rudimentary coverage of audit history achieved,
focus will now shift to briey examining the traditional statutory audit
and envisioning how it might ultimately evolve into the future audit.
THE TRADITIONAL AUDIT
Following the initial establishment of a contractual arrangement between
the auditor and auditee, an audit engagement typically proceeds with a
risk assessment and formulation of an audit plan delineating the scope
and objectives of the audit. Following this, auditors collect and analyze
audit evidence and form opinions pertaining to internal controls as well
as reliability of the information provided by management. At the
engagement conclusion, auditors present a formal report expressing their
opinion. In fact, this approach reects the twentieth century
methodology whereby there are high costs and signicant time delays
associated with information collection, processing, and reporting.
However, these historical costs and delays are often not the norm today.
Most likely, in the current business realm, transactions are often entered
and aggregated such that they can provide near immediate feedback to
relevant stakeholders. Furthermore, academicians and practitioners alike
recognize this information shift and developed numerous solutions that
more appropriately reect the current business environment.
76
ESSAY 3: EVOLUTION OF AUDITING
AUTOMATING THE AUDIT
Organizations historically accustomed to manual audit procedures may
benet from pursuing the future audit in an incremental manner. Such an
approach would basically result in conducting a pilot study to ascertain
the potential benets of audit automation. Because resistance to change is
a universal phenomenon, gradual and careful advancement will likely be
a more tractable approach. Moving forward, this might ultimately result
in greater subsequent support for expansion of automated audit practices
and programs and could signicantly improve the chances of success in
eventually reaching the future audit.
Lanza (1998) argues that low cost solutions for achieving an initial
automated audit experience include introductory CAATS that facilitate
data extraction, sorting, and analysis procedures. These programs require
little training, have no le size limitations, provide detailed audit logs for
use as work paper documentation, and allow for the creation of
auditor-specied reports that may be applied to current and future data
sets. These tools should be initially used to replace manual audit activities
because these are areas where the most substantial benets might be
accrued. For example, the programs could be congured to address tasks
such as footing ledgers, choosing statistical samples, generating
conrmations, and detecting suspicious transactions. In addition, such
tools are capable of testing 100 percent of the records included in a le;
this is a marked improvement over the sampling techniques historically
found in the traditional manual audit. Through these programs, auditors
are able to obtain a better understanding of business operations as well as
enhanced levels of expertise and professional skepticism.
In terms of disadvantages, tools in this category do not operate on a truly
continuous basis. Specically, they are batch process programs activated
periodically according to the audit plan. As such, although they certainly
offer the functionality to improve audit quality, it may eventually be
desirable to consider other methods that more closely align with the
future audit.
In addition to the preceding software considerations, training issues
should be addressed during the process of automating the audit function.
For example, Curtis and Payne (2008) argue that although CAATS are
capable of improving the efciency and effectiveness of auditing
functions, such tools tend to be underutilized. Accordingly, properly
constructed and executed training programs may facilitate more
complete adoption and usage of CAATS by practitioners (Janvrin et al.
2008). Adequate training will be an essential component of any audit
automation initiative in order to optimize the likelihood that auditing
staff will take full advantage of the benets that automated tools can
provide.
77
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
A strategically formulated and implemented plan that includes careful
consideration about issues of resistance, cost and benet tradeoffs,
project scope, and training should result in more favorable outcomes. At
a minimum, CAATS have the potential to serve as a bridging mechanism
between the manual audit and the ultimate future audit. If implemented
and utilized as intended, signicant gains will be realized such that rms
should be more open to entertain the notion of venturing further into the
arena of automation.
THE FUTURE AUDIT
As previously mentioned, basic CAATS contain capabilities to enhance
audit effectiveness and efciency. However, they do not operate on a
24/7 basis and therefore fail to construct a truly continuous auditing
environment whereby exceptions and anomalies may be identied as
they occur. Alternatively stated, they do not work with real-time or close
to real-time data streams and, thus, are not able to address questionable
events such as potential fraud or irregularities in an optimized fashion.
Cangemi (2010) argues that, given the recent advances in business
technologies, the continuing emphasis on the backward looking audit is
simply an outdated philosophy. Instead, he believes that real-time
solutions are needed. As such, rms that successfully experiment with
the CAATS described previously should give eventual consideration to
more advanced programs which contain functionalities resembling the
audit of the future and provide a higher level of assurance.
Fortunately, recently proposed solutions better satisfy this vision. In
general, the programs in this category contain the capabilities to
continuously capture exceptions and outliers in data sets from disparate
systems, provide information and alerting mechanisms to relevant
personnel in an ongoing manner, and essentially confront issues such as
fraud, errors, and misuse of resources in real-time. Furthermore, these
programs may assist in optimizing the audit function by analyzing all
nancial transactions as they occur. As such, this proactive approach
increases efciency and effectiveness in discovering problems and
opportunities for business improvement. However, prior to moving into
this more elaborate domain, additional considerations relative to
business operations are warranted.
In conjunction with this position, Teeter and Vasarhelyi (2011) explain the
optimal alignment of enterprise data and audit procedures. For example,
they mention that manual data corresponds to manual auditing methods.
They also indicate that organizational data that is not strictly manual
may be subject to automated audit procedures on some level. Therefore,
the more manual data an entity maintains, the less it might initially
78
ESSAY 3: EVOLUTION OF AUDITING
benet from audit automation. In order to determine the potential utility
of a robust auditing system, an organization should rst consider the
extent to which its data is automated. Following this, identied manual
enterprise data might reasonably be converted to a more automated state
prior to implementation of tools for automating the audit process.
In moving toward the future audit, the extent to which data, controls,
and processes are automated must be considered. A company that is
overburdened by manual audit processes will need to confront this issue
at some point if the objective is to yield optimal benets from the future
audit. Essentially, if the organization automates its data, controls, and
processes in a manner that properly aligns with the functionalities of the
technology being implemented, the business will likely be in a position to
optimize audit quality.
An enterprise that moves toward greater automation relative to data,
processes, controls, and monitoring tools begins to naturally structure
itself for the coming of the future audit. Given the recent advent of the
real-time economy, this positioning is critical. For example, the
Continuous Audit Monograph (CICA/AICPA 1999) notes that the
development of the digital economy has facilitated a demand from
decision makers, such as potential investors and creditors, for more
timely notication on a wide array of information topics extending well
beyond the traditional nancial statements. Therefore, if these decision
makers require a more continuous information stream on which to
formulate decisions, they will also demand independent assurances
about the reliability of that information. Consequently, the need for a
24/7 auditing protocol becomes apparent if rms intend to compete for
scarce resources and ultimately succeed in the current and evolving
real-time global economy.
With this in mind, one could argue that the traditional manual and
retrospective audit is becoming an untenable position. Also, it could be
argued that the use of rudimentary CAATS such as those described
earlier will eventually be questioned in terms of audit utility. Fortunately,
the idea of the future audit is not a recent phenomenon and there are a
variety of methodologies that have been proposed to reach this plateau.
Embedded Audit Modules
The embedded audit module (EAM) approach involves the installation of
les or code segments within the host system (Groomer and Murthy
1989). For example, in the integrated test facility (ITF) method, a series of
auditor-developed "dummy" master les are instantiated in the live client
system and test transactions are entered as desired by the auditor. These
records are then processed such that only the auditor-created master les
79
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
are affected. Another example in the EAM domain involves a block of
program code that is created and inserted within the client’s system code
structure. Under this scenario, the EAM subsequently monitors
transactions occurring on the host in accordance with the construction of
the code block. When a suspicious item is identied, relevant event
information is recorded in a log that the auditor reviews on an ongoing
basis. Although these approaches have been proposed for a number of
years, several problems have resulted in a lack of acceptance within the
auditing community. For example, Groomer and Murthy (1989) point out
that the EAM method may reduce client system performance, create
excessive data sets relative to the event log, and be subject to code
modication by astute programmers. Because of such issues with the
embedded approach, it currently exists as primarily an academic topic.
Monitoring and Control Layer
The monitoring and control layer (MCL) architecture is considered a
CAAT that may aid in providing continuous monitoring and control of
accounting information systems (Debreceny et al. 2005). Vasarhelyi,
Alles, and Kogan (2004) initially introduced the MCL architecture as an
alternative to the EAM methodology. In particular, several researchers
have pointed out that, in contrast with EAM, MCL has fewer concerns
related to software maintenance, legal liability, client independence, and
reliance on enterprise personnel (Alles et al. 2006; Kuhn and Sutton 2010).
In terms of functionality, Best, Rikhardsson, and Toleman (2009) indicate
that MCL is essentially a self-governing, middleware solution that
extracts data from systems and conducts appropriate analyses as desired.
The primary function of the MCL method is to continuously analyze and
compare data obtained against specic benchmarks or other criteria.
When exceptions are noted, alerts are generated and sent to the auditors
for review and investigation. Consequently, the MCL approach is
preferable to the EAM methodology on many dimensions, including
mutual exclusivity of the auditing module and client system(s).
However, although the MCL approach is superior to the EAM
techniques, it is still perceived as a suboptimal solution. For example,
Sigvaldason and Warren (2004) indicate that many enterprises maintain a
variety of disparate systems and this presents substantial difculties and
challenges in establishing the required connections between the MCL and
various client systems themselves. Also, given its inherent status as a
monitoring and control solution, some might argue that the maintenance
of auditor independence in the MCL environment is inherently
problematic. Whatever the case, much like EAM, the MCL approach has
not yet received widespread acceptance in practice.
80
ESSAY 3: EVOLUTION OF AUDITING
Audit Data Warehouse
The audit data warehouse model has been offered as a viable future audit
solution. In particular, this approach appears to alleviate the problems
and concerns associated with both the EAM and MCL techniques. By
denition, a data warehouse is "a big data pool—a single, company-wide
data repository—with tools to extract and analyze the data" (David and
Steinbart 1999, 30). Essentially, a data warehouse is linked with the
various and disparate enterprise systems such that it readily accepts and
integrates the pertinent data being generated throughout the
organization (Rezaee et al. 2002). In addition, the data warehouse may be
incorporated with data marts, which are a set of smaller, focused
warehouses in which each addresses a particular functional area such as
accounting or marketing. Furthermore, the audit warehouse and data
mart(s) may reside on the same audit server.
From an operational perspective, enterprise data is extracted, converted,
standardized, and installed in an ongoing manner within the data
warehouse context. In addition, each data mart gathers, transforms, and
loads appropriate data from the warehouse according to specications
and congurations. Also, each data mart contains various standardized
audit tests that operate at stipulated time intervals (for example,
continuously, daily, weekly), collect audit evidence, and generate
exception reports for auditor review and investigation.
A conceptual model that utilizes the audit warehouse architecture is
AuSoftware. According to Sigvaldason and Warren (2004), it accumulates
necessary data on a continuous basis in at le structures from a
disparate array of organizational systems (for example, ERP, legacy,
outsourced). To minimize processing burden, AuSoftware imports data
in read only format into a data warehouse or "audit data mart" that
provides for continuous auditing procedures. In addition, as suspicious
items are identied, the software is able to communicate control and
audit alerts via Web-based interfaces or more direct routes such as cell
phones. AuSoftware has the capability to identify anomalies and
irregularities on a 24/7 basis and alert auditors in an immediate manner
such that interventions may occur in a timely fashion. This is a signicant
improvement over the traditional audit that simply evaluates a small
sample of historical transactions and items on a periodic basis and may
either fail to identify problems that exist or detect problems too late for
adequate resolutions to be implemented.
Audit Applications Approach
A very recent development entails the usage of specic applications or
"apps" in conducting the future audit. The AICPA Assurance Services
Executive Committee (Zhang et al. 2012) has promoted the idea that a
81
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
standardized set of data
2
from multiple cycles be used by a series of audit
apps that might be constructed and procured in alignment with audit
plans and assertions in order to effectively perform the future audit. For
example, for the audit activity "evaluate aging of accounts receivable," an
audit app could be utilized to query accounts receivable transaction
details, compare percentages in all aging categories with prescribed
industry standards, and alert auditors when the actual percentages vary
signicantly from the designated standards. Furthermore, additional
apps could be created and otherwise obtained as required for completing
remaining audit activities in fulllment of the organizational audit plan
and assertions.
Other Future Audit Considerations
The preceding discussion demonstrates that sophisticated audit
technologies are being actively researched and developed to facilitate the
future audit. However, many organizations will have much to overcome
prior to moving toward that realm. For example, the CICA/AICPA (1999)
formulated the following listing of six conditions necessary for
advancing to the future audit:
r
Subject matter with suitable characteristics. Highly automated
processes are needed to provide reliable information shortly after
occurrence of associated events and transactions.
— Business has progressed substantially in providing close to
real-time information for key processes. Their utilization for
audit is still spotty.
r
Reliability of systems providing the subject matter. Probability the
system will operate effectively over a given period of time;
reliability optimized when enterprise controls are effective and
system provides complete and accurate information in a timely
fashion.
— Although SysTrust has been out for a decade, it is only now
that there is more attention given to assurance on system
reliability. This attention is also spotty.
r
Audit evidence provided by highly automated procedures. Auditors
must quickly understand causes of all recognized anomalies and
errors, determine where they originated, and discuss corrective
action with management.
— We have not yet managed to provide and use real-time audit
evidence.
2
The audit data standard predicts a series of at (or tagged) standard les that are to be provided
by companies to internal and external auditors. The general ledger and receivables standards were
exposed by the AICPA and are under revision as of the publication date of this paper.
82
ESSAY 3: EVOLUTION OF AUDITING
r
Reliable means of obtaining results of audit procedures on a timely basis.
The outcomes of automated audit procedures must be efciently
communicated to auditors; this suggests reliable and efcient
electronic communication methods with appropriate security
measures in effect.
— As discussed in essay 2, "The Current State of Continuous
Auditing and Continuous Monitoring," the external audit
profession has not yet adopted "close to the event" audit
technologies, although they are in the process of advising
internal audit departments on how to do so.
r
Timely availability of and control over audit reports. Organizational
information and associated audit reports must be available in an
ongoing manner and easily accessed by legitimate users.
— Substantive adoption of automated work papers, audit
warehouses, and corporate internal report distribution has
drastically reduced report distribution challenges.
r
High degree of auditor prociency in information technology and the
audited subject matter. Auditor must have necessary skill sets to
handle the engagement.
— Pockets of practitioners developed IT skills. Recently there is
growing awareness of the need to increase auditor IT and
analytic prociencies.
Therefore, a host of variables and characteristics must be adequately
addressed in order to fully realize the benets of the future audit.
Although the system architecture and software components are
extremely important considerations, complementary elements such as
auditor education, the socio-technical environment of the rm, and tone
at the top are fundamental as well. Consequently, comprehensive
strategic planning that joins technical issues with human issues is also a
necessary ingredient in helping to ensure a successful transition to the
future audit.
CONCLUSION
Auditing has made great strides in the past decade, but it has not
seemingly kept pace with the real-time economy. Some auditing
approaches and techniques that were valuable in the past now appear
outdated. Also, the auditing evolution has reached a critical juncture
whereby auditors may either lead in promoting and adopting the future
audit or continue to adhere to the more traditional paradigm in some
manner. Future audit approaches would likely require auditors,
regulators, and standards setters to make signicant adjustments. Such
83
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
adjustments might include (1) changes in the timing and frequency of the
audit, (2) increased education in technology and analytic methods, (3)
adoption of full population examination instead of sampling, (4)
re-examination of concepts such as materiality and independence, and (5)
mandating the provisioning of the audit data standard. Auditors would
need to possess substantial technical and analytical skills that are
currently not components of most traditional four year university
accounting programs.
SOX introduced the rst major change in the mandate of the public
company audit. This new prescription focuses on auditor assessment of
internal controls, a very important step in the assurance of future systems
that will be modular, computerized, and often outsourced. The
accounting profession now faces an opportunity to further elevate the
audit to a higher level of automation. It is imperative that accountants
ultimately lead the way in adoption and implementation of the future
audit such that they continue to be the professionals of choice relative to
audit engagements of the future.
REFERENCES
AICPA Assurance Services Executive Committee, June 2011. Audit Data Standards
and Apps. University Presentation.
Alles, M., Brennan, G., and Kogan, A., 2006. "Continuous Monitoring of Business
Process Controls: A Pilot Implementation of a Continuous Auditing System at
Siemens," International Journal of Accounting Information Systems 7 (2): 137–161.
Association of Certied Fraud Examiners, 2010. Report to the Nations on
Occupational Fraud and Abuse.
Best, P., Rikhardsson, P., and Toleman, M., 2009. "Continuous Fraud Detection in
Enterprise Systems Through Audit Trial Analysis," Journal of Digital Forensics,
Security, and Law 4 (1): 39–60.
Cangemi, M., and Singleton, T., 2003. Managing the Audit Function: A Corporate
Audit Department Procedures Guide, 3rd ed. John Wiley & Sons, Inc.
Cangemi, M. April, 2010. "Internal Audit’s Role in Continuous Monitoring," The
EDP Audit, Control, and Security Newsletter 41 (4).
Chandler, A. D., Jr., 1977. The Visible Hand: The Managerial Revolution in American
Business. Cambridge, Massachusetts: Harvard University Press.
CICA/AICPA Study Group, 1999. Research Report: Continuous Auditing. Toronto,
Canada: The Canadian Institute of Chartered Accountants, American Institute
of Certied Public Accountants.
Curtis, M., and Payne, E., 2008. "An Examination of Contextual Factors and
Individual Characteristics Affecting Technology Implementation Decisions in
Auditing," International Journal of Accounting Information Systems 9: 104–121.
David, J.S., and Steinbart, P.J., December 1999. "Drawing in Data," Strategic
Finance. 30–36.
Davis, G., 1968. Auditing & EDP. New York, New York: American Institute of
Certied Public Accountants, Inc.
84
ESSAY 3: EVOLUTION OF AUDITING
Debreceny, R., Gray, G., and Yau, W., 2005. "Embedded Audit Modules in
Enterprise Resource Planning Systems: Implementation and Functionality,"
Journal of Information Systems 19 (2).
Groomer, S. M., and Murthy, U. S., 1989. "Continuous Auditing of Database
Applications: An Embedded Audit Module Approach," Journal of Information
Systems 3 (2): 53–69.
Janvrin, D., Lowe, D., and Bierstaker, J., 2008. Auditor Acceptance of
Computer-Assisted Audit Techniques. Working Paper.
Kuhn, R.J., and Sutton, S.G., 2006. "Learning from WorldCom: Implications for
Fraud Detection Through Continuous Assurance," Journal of Emerging
Technologies in Accounting 3: 61–80.
Kuhn, R.J., and Sutton, S.G., 2010. "Continuous Auditing in ERP System
Environments: The Current State and Future Directions," Journal of Information
Systems 24 (1): 91–112.
Lanza, Richard, 1998. "Take My Manual Audit, Please!" Journal of Accountancy
33–36.
Moussalli, Stephanie, October 2005. "Accounting for the Journal’s First 100 Years:
A Timeline from 1905 to 2005," Journal of Accountancy.
Rezaee, Z., Sharbatoghlie, A., Elam, R., and McMickle, P., 2002. "Continuous
Auditing: Building Automated Auditing Capability," Auditing: A Journal of
Practice and Theory 21 (1): 147–163.
Seidler, L.J., Andrews, F., and Epstein, M.J., 1977. The Equity Funding Papers: The
Anatomy of a Fraud. New York: John Wiley & Sons.
Sigvaldason, T., and Warren, J.D., 2004. Solving the Software Architecture Riddle to
Deliver Enterprise-wide Continuous Financial Process Monitoring and "Auditing."
Financial Market Solutions, LLC.
Teeter, R., and Vasarhelyi, M., June 2011. Audit Theory and Assurance Automation.
Rutgers University Presentation.
Vasarhelyi, M., Alles, M., and Kogan, A., 2004. "Principles of Analytic Monitoring
for Continuous Assurance," Journal of Emerging Technologies in Accounting 1(1):
1–21.
Zhang, L., Pawlicki, A. R., McQuilken, D., and Titera, W. R., Spring 2012. "The
AICPA Assurance Services Executive Committee Emerging Assurance
Technologies Task Force: The Audit Data Standards (ADS) Initiative," Journal of
Information Systems 26 (1): 199–205.
85
ESSAY 4
Reimagining Auditing
in a Wired World
1
Paul Eric Byrnes, CMA
Thomas R. Criste, CPA
Trevor R. Stewart, CA, PhD
Miklos Vasarhelyi, PhD
OVERVIEW
How would nancial statement audits be designed if auditing were a
new service that had just been invented? There can be little doubt but
that audit processes would be designed from the get-go to make optimal
use of today’s amazing technology in order to enable auditors to provide
the most effective and efcient service possible within the bounds of
economic viability. Instead, for the most part, auditors use legacy
processes that are not much different from those of 50 years ago except
that they have been computerized. The emphasis has been on improving
efciency, and although effectiveness has improved as well, there has not
been the quantum leap that technology can enable.
Our thesis is that the profession needs to achieve that quantum leap. This
will involve deconstructing and re-engineering processes; researching
how data science and related technologies can be harnessed and tailored
into applications for auditors; extending auditing theory to encompass
new approaches; modifying auditing standards where necessary and
providing plenty of new guidance; and using today’s ubiquitous
computing and connectivity to transform where and how work gets done
and to enable continuous auditing. We use a "blue sky" scenario to
describe what future reporting and auditing systems might resemble and
we discuss how technology could be used to transform auditing.
1
First published 2014. From the AICPA Assurance Services Executive Committee (ASEC) Emerg-
ing Assurance Technologies Task Force.
87
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
INTRODUCTION:BLUE SKY SCENARIO
It was 7:30 in the morning on Tuesday, June 17, 2020, and Sally was just
settling in for another day as the external audit partner of ML
Enterprises, Inc. (MLE), a $12 billion U.S. construction company
operating in over 30 countries. As usual, her rst task after getting her
skinny latte was to sign in to AART, the audit rm’s Automated Audit,
Reporting, and Tracking system. AART was developed to leverage the
widespread availability of information on a 24/7 basis so that technology
could continuously monitor MLE’s controls, transactions, and account
balances. AART had been initially congured and customized for MLE
business operations and technology platforms, but was also capable of
learning over time. In that sense, AART was able to better understand the
client’s business. This offered the dual benets of reducing the number of
legitimate events agged for auditor review (false positives), as well as
identifying trends, anomalies, and patterns that Sally and her team had
not explicitly addressed or considered.
As she looked at the AART dashboard that she had customized for her
use, the majority of her audit status indicators were solid green.
However, she noticed a ashing red indicator relating to the Hong Kong
Treasury Operations group, and a yellow warning icon associated with
the Brazilian operations. She had also been copied on a message that
AART sent to her controls team notifying them of a modication to a key
control parameter in the centralized SAP system. She clicked on the red
indicator, and saw that an unusually large transfer had been executed in
Hong Kong last night between the Asian and European regional
operations. She then forwarded the pertinent information to her partners
in Hong Kong and Munich, and set a ag to remind her to review their
responses the following day. In addition, she transmitted the
Brazil-related information to one of her managers for follow-up activities.
Sally then examined the Daily Transaction Testing Report from the
previous day. This was a randomly generated list of transactions chosen
for testing in addition to those specically identied by the AART
system. Of the 12 transactions selected yesterday, 10 had already been
reviewed and closed by the audit team, and two remained open. She
reviewed the tests performed and resulting conclusions, and was satised
that the evidence and explanations provided were appropriate and
sufcient. She also reviewed the status of the two open items and noted
that management had promised to respond by close of business today.
After meeting with the controller, she returned to her ofce at 11 a.m. She
realized that the continuously audited nancial statements would be
posted on MLE’s website at noon, as they were every day, and she
wanted to conrm whether there were any signicant unresolved issues
requiring prior attention. After logging in to AART, she noticed a
88
ESSAY 4: REIMAGINING AUDITING IN A WIRED WORLD
message from her controls specialist indicating that the SAP password
complexity parameter had been updated to require at least one number
and one special character, and he believed this change to be an
improvement. He had previously discussed this matter with the chief
security ofcer, who provided evidence of the policy change and the
related approvals. Sally then reviewed a number of statistical reports,
charts, and graphs produced by AART and concluded that the nancial
statements could be published and the audit opinion would remain
unqualied. Although the Hong Kong transaction had not yet been
veried, there was nothing to indicate that it was improper, and, in the
worst case, would result in an adjustment the following day.
Sally then recalled that she had a 3:30 p.m. meeting next Monday with
representatives from Future Financial (FF) to approve guidelines of an
evolving lending arrangement with MLE’s U.S. operations. Negotiations
were being nalized relative to a line of credit with FF, and part of this
nal discussion was to involve associated covenant terms. Specically, FF
was looking for regular access to particular information in addition to
associated independent assurances. As Sally reected upon this proposal,
she theorized that it would not present any insurmountable challenges.
Essentially, MLE was historically proactive, and, as such, recognized long
ago that a wider variety of stakeholders were increasingly demanding
access to a diverse mixture of real-time information with attached
assurances. Among other things, the AART system was designed to
facilitate this stakeholder-centric approach. Consequently, she envisioned
a rather simple solution. First, a Web-based dashboard would be
constructed specically for FF, and it would contain all of the required
information and metrics relative to the lending arrangement. Second, this
page would be dynamically refreshed on a daily basis immediately
subsequent to the posting of nancial statements. Third, FF would be
assigned permissions and issued a user name and password combination
(or set of combinations) to access, in an ongoing manner, the formulated
website, as well as the area(s) containing relevant assurances. Sally
actually viewed this as a type of pilot project, because bondholders and
other lenders were starting to request information on a similar platform.
She knew it was only a matter of time that such provisioning of
information would become the norm. Whatever the case, Sally was
meeting with her team later today to construct an action plan concerning
the issue with FF, and was condent that a suitable solution would be
established.
On a more challenging level, rumblings were beginning to be felt on a
global basis relative to corporate social responsibility concerns. In
particular, environmental groups and community members were placing
increasing pressure on MLE to provide relevant information in more of a
real-time format. Although Sally felt condent that this could ultimately
be accomplished via the AART system, she wondered about the
89
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
complexity of maintaining such quantitative and qualitative information
on a current basis. Like most companies, MLE presently publishes an
annual sustainability report that outlines key initiatives and metrics
about social responsibility at MLE, and it is historically a very challenging
and time-consuming project. Although she believes that the quantitative
portion of reporting could be readily presented on a more frequent basis,
she envisions signicant coordination barriers in maintaining the
independently assured qualitative information in a comparable manner,
largely because of MLE’s international presence. Not surprisingly, Sally
feels fortunate that this is an issue not requiring an immediate resolution.
Nevertheless, she has scheduled an initial brainstorming session with her
team in an effort to begin articulating potential operational strategies.
On a lighter note, Sally had previously arranged for lunch to be delivered
to celebrate the 33rd birthday of one of her team members, Rob, who was
her construction accounting specialist. He had 4 years of public
accounting experience, including 2 years with the SEC, and was an
assistant controller in another construction company for 6 years. Rob’s
background was indicative of the experience level of her team in general.
The junior member was Allison, who had a Ph.D. in statistics as well as 7
years of experience at a major insurance company. The audit specialist,
Trevor, was a CPA with 9 years of experience and was responsible for the
conguration of the AART system, as well as the design and execution of
all substantive audit procedures. Rounding out the team were Subrata,
the controls specialist who came to the rm after 8 years in IT and
internal audit, and Jorge, who was a construction industry analyst for 17
years at Deutsche Bank. Sally reected back to the day she was hired
directly out of business school, and noted how AART had radically
changed the stafng model and audit dynamics. The audit was now
being conducted with a handful of highly experienced and
well-compensated specialists. In addition, having a skillset that included
extensive technical and analytical expertise was no longer optional.
Essentially, there was little demand at MLE for an inexperienced,
traditionally educated accounting graduate.
Although the preceding anecdote may be perceived as visionary, it is
nevertheless a window into what the future audit might entail. As the
new economy continues to evolve, and stakeholder groups progressively
seek access to more timely information, the audit and reporting models
will need to adapt in accommodating this landscape. Given this, the
balance of this chapter will be primarily devoted to presenting issues,
observations, and potential challenges relevant to audit theory, process,
and technology. In doing so, emphasis will be placed upon offering
preliminary insight concerning how these items might be addressed to
better meet the future needs of stakeholders. In conclusion, future audit
90
ESSAY 4: REIMAGINING AUDITING IN A WIRED WORLD
technology considerations will be briey explored to develop an
improved overview of how the audit might evolve in the coming years.
USING TECHNOLOGY TO TRANSFORM
AUDITING
Although auditors embrace and make extensive use of information
technology, little has been done to consider how auditing might be
transformed by it. For the most part, IT has been used to computerize
and improve the efciency of established processes rather than transform
or replace them. Consequently, improvements have been incremental
rather than transformative. We will discuss some of today’s IT enablers
and their potential for improving audits.
Technology Enablers
Thanks to the Internet and exponential advances in core technologies,
today’s auditors practice in a globally connected world of ubiquitous
computing and communications devices that collectively provide a
platform for transformational applications. Smartphones, tablets, and
other mobile computing and communications devices are pervasive and
always on, and information workers are no longer tethered to ofce
desks but work from home, coffee shops, public libraries and parks,
airport terminals and airplanes, and from rent-by-the-hour ofce suites
(AICPA 2012).
In years past, auditors worked in relatively isolated local teams from the
same ofce, but today’s auditors are able to operate more uidly,
connecting to teams from wherever they happen to be, as seamlessly as if
they were in the same room, and individual skills can be leveraged
globally across many audit engagements. A statistical specialist in
Amsterdam can participate in audits conducted in Adelaide or Ankara.
Many audit procedures today can be deconstructed into tasks that can be
performed wherever is most effective, whereas in years past, audit
procedures had to be performed onsite by vertically integrated audit
teams of local ofce resources. For example, the onsite, client-facing audit
team can focus on tasks that only they can perform (such as observing the
performance of internal controls or meeting with the CFO), and
outsource back-end tasks that may be better performed remotely by
teams of specialists or third-party providers. For example, as we will
describe later, the mechanics of bank conrmations can be performed by
third-party organizations that specialize in that mundane but critical task
and can perform at the highest standards of reliability and
91
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
security—higher than is likely from the generalist client-facing team.
Analytical procedures or journal entry testing for audits in Boston or
Budapest can be performed better by a specialist team in Bangalore that
performs the procedure day in and day out for dozens of audit
engagements. This Internet-enabled deconstruction of tasks into separate
processes that are performed wherever it is most effective and is mirrored
by similar developments in computing.
Cloud computing is one example of how tasks are deconstructed into
separate processes that migrate over the Internet to where they can be
performed most effectively. Rather than operate its own IT infrastructure
and software, an entity effectively plugs into an IT utility that provides
and maintains the necessary software and manages and stores data. End
users may need nothing more than browsing software. Thus, the total
task is deconstructed into a simple front-end, and a back-end somewhere
in the cloud, where a massively equipped provider does the heavy lifting
for hundreds of entities.
Data science and related technologies have advanced enormously in
recent years, incorporating theories, techniques, and software
applications from many elds, including data analysis, business
intelligence, mathematics and probability, statistical learning including
pattern recognition, data visualization, gamication, big data analytics,
and text and process mining. Applications from the world of data science
can be applied by auditors to perform more effective audits and to
provide new forms of audit evidence not previously available to
practitioners (Hoogduin, Yoon, and Zhang 2014). Using new applications
effectively requires learning new skills and the support of specialists,
which can be enabled by the Internet.
AUDIT OPPORTUNITIES
The technology enablers discussed in the previous section provide
opportunities for signicantly improving audit effectiveness as well as
efciency. In this section we discuss some of those opportunities and
their implications for the profession.
More Effective Audit Data Analytics
Audit data analytics (ADA) is the science and art of discovering and
analyzing patterns, identifying anomalies, and extracting other useful
information in data underlying or related to the subject matter of an audit
through analysis, modeling, and visualization for the purpose of
planning or performing the audit. ADA includes methodologies for
identifying and analyzing anomalous patterns and outliers in data;
92
ESSAY 4: REIMAGINING AUDITING IN A WIRED WORLD
mapping and visualizing nancial performance and other data across
operating units, systems, products, or other dimensions for the purpose
of focusing the audit on risks; building statistical (for example,
regression) or other models that explain the data in relation to other
factors and identify signicant uctuations from the model; and
combining information from disparate analyses and data sources for the
purpose of gaining additional insights.
ADA includes, but is not limited to, analytical procedures: preliminary
analytical procedures used for planning (AU-C sec. 315); substantive
analytical procedures used for substantive testing (par. .05 of AU-C sec.
520); and analytical procedures performed near the end of the audit to
assist the auditor when forming an overall conclusion about whether the
nancial statements are consistent with the auditor’s understanding of
the entity (par. .06 of AU-C sec. 520). ADA also includes traditional le
interrogation. The scope of ADA is illustrated in gure 4-1.
Figure 4-1: ADA includes but is not limited to traditional analytical
procedures and le interrogation.
The data analytics literature distinguishes between two different modes
of analysis, exploratory and conrmatory (Tukey 1977), and we continue
that distinction here in the context of ADA. Exploratory ADA is
bottom-up and inductive. It starts with the data and the auditor asking
questions such as, "What does the data suggest is happening? Does the
data suggest something might have gone wrong? Where do the risks
appear to be? Are there potential fraud indicators? On what assertions
should we focus? What models and approaches appear to be optimal for
analytical procedures?" Exploratory ADA is most useful in audit
planning—understanding the entity and its environment, identifying and
assessing the risks of material misstatement, and designing further audit
procedures. Conrmatory ADA, on the other hand, is top-down and
deductive. It starts with audit objectives and assertions. It tends to be
model-driven with the auditor asking questions such as, "Is the subject
93
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
matter consistent with my model (that is, with expectations)? Are there
deviations that are individually signicant or that form a pattern, such
that they indicate the potential presence of material misstatement?"
Conrmatory ADA is used to provide the auditor with substantive or
controls assurance about whether management’s assertions are
materially correct—ultimately, whether the nancial statements are free
from material misstatement.
2
The use of visual exploratory techniques can help auditors see patterns,
trends, and outliers that are otherwise hidden, and reveal relationships
between variables that could be the foundation for a conrmatory model.
Conrmatory techniques are more formal and tend to be more
mathematical and analytical (Behrens 1997); for example, they might
utilize multiple regression analysis or the extraction and summarization
of transactions meeting certain risk criteria. However, there is no bright
line distinction between exploratory and conrmatory ADA, and they
tend to be used iteratively. For example, initial exploratory techniques
may suggest a fruitful conrmatory model to be used for substantive
analytical procedures, but the residuals from that model (actual minus
expected) may lead to the discovery of additional factors that can be used
to improve the model. Some of the same techniques can be used for
exploratory and conrmatory analytics.
In the audit of nancial statements in accordance with generally accepted
auditing standards, there are numerous potential opportunities for
making use of ADA. These include the following:
r
Identifying and assessing the risks associated with accepting or
continuing an audit engagement (for example, the risks of
bankruptcy or high-level management fraud).
r
Identifying and assessing the risks of material misstatement
through understanding the entity and its environment (AU-C sec.
315). This includes performing preliminary analytical procedures
as well as evaluating the design and implementation of internal
controls and testing their operating effectiveness.
r
Performing substantive analytical procedures in response to the
auditor’s assessment of the risks of material misstatement (AU-C
sec. 520).
r
Identifying and assessing the risks of material misstatement of the
nancial statements due to fraud, and testing for fraud having
regard to the assessed risks (AU-C sec. 240).
2
Tukey (1977) draws an analogy with the processes of Anglo-Saxon criminal justice where there
is a clear divide between the search for evidence, which is the responsibility of the police and other
investigative forces, and the evaluation of the strength of evidence and degree of guilt, which is the
responsibility of the courts. Exploratory data analysis is detective in character; conrmatory data
analysis is judicial or quasi-judicial in character.
94
ESSAY 4: REIMAGINING AUDITING IN A WIRED WORLD
r
Performing analytical procedures near the end of the audit to
assist the auditor when forming an overall conclusion about
whether the nancial statements are consistent with the auditor’s
understanding of the entity (AU-C sec. 520).
More Assurance
The auditor’s overall objective is to obtain a reasonably high level of
assurance about whether the nancial statements are free from material
misstatement. Reasonably high is not dened, but is commonly
understood to mean no less than 95 percent condence, where degree of
condence is a measure of the auditor’s degree of subjective professional
belief rather than some objectively calculable probability. Technology can
be used to achieve the same level of assurance but more efciently at a
lower cost, or it can be used to achieve a higher level of assurance via a
more effective audit at similar cost. Technology also enables statistical
techniques (for example, sampling and regression analysis) that can
provide objectively quantiable condence levels to help build assurance.
Economics has driven auditors to focus mostly on improving efciency
(achieving the same level of assurance but at lower cost). Less attention
has been paid to increasing assurance at the same cost by improving
effectiveness, even though that cost would buy the additional benets of
better meeting client and investor expectations and of reducing audit and
reputational risk and liability. In medicine, physicians are expected to use
better technologies as they come along if they signicantly improve
patient outcomes at reasonable cost. In auditing, professional standards
should encourage auditors to consider and use technologies that increase
assurance beyond the minimum required where economically feasible.
Professional standards need to be technology agnostic, but that does not
mean that they should not encourage auditors to make the best use of
technology to perform the best possible economically viable audits.
An example of where technology can and should be used to increase
assurance is in detailed tests of transactions and balances. Traditionally,
such tests were performed on a small sample of items. This was the only
way to do it when items had to be selected from a printed or
hand-written listing. With computerized data and le interrogation audit
software, however, many tests can be performed on 100 percent of the
population. It is also possible to simultaneously analyze and visualize the
complete population in ways that can reveal unexpected patterns and
outliers worthy of special investigation. For certain procedures, sampling
is still necessary (for example, the physical inspection or third-party
conrmation of assets, or the analysis of complex contracts).
Nevertheless, even where sampling is necessary for certain essentials, it
is often possible to increase audit assurance at little additional cost by
analyzing and performing other procedures on the entire population.
95
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Auditing With Big Data
3
Big data is the product of a technological environment in which almost
anything can be recorded, measured, and captured digitally, and thereby
turned into data. The process, often called "datacation," may track
thousands of simultaneous events; be performed in real-time; involve
numbers, text, images, sound, and video; and require petabytes of
storage capacity. Big data has been used in marketing to target potential
customers, in political campaigning to study voter demographics, in
sports to evaluate teams and players, in national security to identify
threats, in biology to study DNA, in law enforcement to identify crime
suspects, in public health to identify epidemics, and in securities
regulation by the SEC to identify a multitude of behaviors including
insider trading and accounting fraud.
Big data analytics is the science and art of improving knowledge about or
gaining insights into some eld of interest or subject matter by
identifying and analyzing related patterns and correlations in big data. In
auditing, the basic subject matter consists of the transactions and
balances that underlie the nancial statements. These usually reside in
the entity’s enterprise resource planning (ERP) and data warehouse
systems and, even if voluminous, do not in themselves constitute big
data within the normal meaning of the term. The audit opportunity is to
use related big data as an auxiliary to the data actually being audited—to
audit with big data, using analytics to identify and analyze patterns and
correlations that reveal matters of audit interest.
There are certain characteristics of big data analytics that are causing
users to rethink data usage. The rst is that it is increasingly possible to
analyze the entirety or almost all data rather than just a small, carefully
chosen subset or sample. This can lead to more robust models. For
example, if an auditor wants to determine what characteristics of journal
entries are indicators of risk of error or fraud, it is possible to analyze all
the journal entries and use this information to identify current journal
entries that are really unusual. In the past, a high degree of care was
necessary to eliminate bad data; now when all the data is available, a
certain degree of pollution is acceptable for many applications. For
example, if a model is based on just a small number of observations, the
auditor must take great care to ensure that they are accurate in order not
to skew the model. If the model is based on a large number of
observations, then the auditor can tolerate some errors because, unless
they are systemic, their effect will be insignicant.
3
See Stewart, T., M. Cao, and R. Chychyla, "Big Data Analytics in nancial Statement Audits,"
Accounting Horizons, forthcoming 2014.
96
ESSAY 4: REIMAGINING AUDITING IN A WIRED WORLD
A second shift in thinking is that instead of trying to understand the
fundamental causes of complex phenomena it is increasingly possible to
identify and make use of correlations. For example, according to
Mayer-Sch
¨
onberger and Cukier
researchers in Canada are developing a big-data approach to spot
infections in premature babies before overt symptoms appear. By
converting 16 vital signs, including heartbeat, blood pressure,
respiration, and blood-oxygen levels, into an information ow of
more than 1,000 data points per second, they have been able to
nd correlations between very minor changes and more serious
problems.
Although these observations may allow doctors to eventually
understand fundamental causes, simply knowing that something is likely
to occur is more important than understanding exactly the reason. It is
analogous to auditing applications in which restatements, accounting
fraud, bankruptcy, or going concern issues are correlated with indicators
obtained from company lings and sources of data. As stated earlier, the
SEC uses big data analytics to identify insider trading and accounting
fraud.
Continuous Auditing, Continuous Assurance
It is possible with today’s technology to continuously monitor and audit
an entity’s transactions in close to real-time, or at least at frequent
intervals. This ability may be used to monitor and assess the operating
effectiveness of automated internal controls, or to perform substantive
tests. Although many internal auditors already do continuous auditing,
at least for some applications, it is still rare among external auditors.
Because internal auditors are part of the entity’s internal control system,
an ability to detect potential problems as soon as they occur is an
enhancement to internal control that should factor into the external
auditor’s evaluation of internal control.
There are at least two ways in which continuous monitoring and auditing
techniques can be directly useful to external auditors. First, such
techniques can alert them to potential problems as early as possible, thus
giving them more time to respond and adapt plans for the remainder of
the audit. This enhances audit quality and client service. Second,
continuous monitoring and auditing can help spread the work effort
throughout the year. This is not necessarily useful in an environment
where the audit team needs to be on the client’s premises, as that
typically involves travel and setup time. However, in today’s connected
world it is possible to monitor and audit remotely. To the extent that this
can reduce workloads and stress during busy season, it will tend to also
97
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
improve audit quality. The ability to use correlation models with big data
in order to pinpoint transactions or events of audit interest becomes
signicantly more useful when applied continuously.
There are many reasons that reporting entities issue audited nancial
statements only once a year, including the cost and effort of gathering,
preparing, auditing, and presenting information. Because ERP systems
update general ledger accounts as transactions are initially recorded, it is
possible to produce nancial statements on a more frequent, almost
continuous basis, and Web-based technologies can make such statements
almost instantly available online. The use of interactive data reporting
standards such as XBRL greatly enhances the appeal and utility of online
reporting. As outlined in the MLE story, today a company could, in
principle, provide condensed nancial statements of some kind on a
daily or even close to real-time basis. If stakeholders demand such
continuous reporting and are unwilling to accept additional information
risk, it is likely that audit assurance will also be required on a continuous
basis. Should that occur, continuous auditing will be essential rather than
optional.
More Effective Fraud Detection
ADA techniques together with the ability to analyze and correlate vast
amounts of data have revolutionized fraud detection. Patterns and
connections that might never have been discovered in the past can be
much more easily identied, analyzed, and visualized. Network analysis,
used to analyze connections and relationships between people or entities,
can be used to identify related parties possibly involved in fraudulent
activities. The SEC is using data analytics applied to big data to look for
inside traders—individual or collusive—and for indications of potential
accounting fraud (Financial Times 2014).
Reducing False Positives
When an entire population is analyzed for anomalies and outliers, it is
possible for a huge number of false positives to be agged, and it is the
fear of being overwhelmed that often leads auditors not to perform such
analyses in the rst place. Although false positives can never be
eliminated entirely, their incidence can be signicantly reduced via
statistical learning and other techniques that enable the identication of
"exceptional exceptions" (Issa 2013). Credit card companies use such
techniques to identify potentially fraudulent transactions without
overwhelming cardholders with false alarms.
98
ESSAY 4: REIMAGINING AUDITING IN A WIRED WORLD
AUDIT PROCESS RE-ENGINEERING:AN
EXAMPLE
Today, many audit processes are essentially unchanged from those
performed decades ago, even though newer technology may be used to
perform them more efciently and opportunities abound for using
technology to reengineer processes so they achieve the same objectives
more effectively. In this section, we illustrate this with account
conrmations—a mundane but critical audit process—that includes
conrmations of bank account balances, accounts receivable, and
accounts payable.
In a traditional conrmation, the auditor selects a sample of accounts to
conrm and then generates and mails letters asking account holders to
conrm the amount. If an account holder does not respond then
alternative procedures are performed.
Although simple in principle, conrmations must be carefully performed
so that the auditor can be sure that the process is not subverted by
fraudulent actors and that requests for conrmation are directed at
parties who are authorized to respond. One way to improve the process
in terms of added security and reduced tedium is to outsource it to an
organization that specializes in conrmations. Typically, the auditor
provides the conrmation service with a list of accounts to conrm. The
service contacts the account and receives the response, and
communicates it back to the auditor, as illustrated in gure 4-2. The value
added by the service provider is (a) an established secure network
including a public key infrastructure that ensures all communications are
secured and digitally signed, thus guaranteeing that communications are
not intercepted or subverted, and that the parties are who they purport to
be; (b) a network of authenticated participating banks or other
organizations that sign up and agree to conrm via the service provider;
and (c) the performance and administration of a mundane tedious task,
thus freeing up audit personnel to focus on higher-level tasks.
When the service provider is asked to conrm an account not in the
network, they attempt to authenticate and add it to the network. Because
the service provider works for many different audit rms the investment
in the network can be leveraged over a large base.
99
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 4-2: Basic connectivity in auditor use of conrmation service in
performing the accounts receivable (A/R) conrmation procedure.
When it comes to accounts receivable, Titera (2013) suggests yet another
approach that could provide greater assurance than a traditional
conrmation in certain businesses. If there is a short average collection
cycle, balances outstanding at the date of the accounts receivable audit
could be matched with subsequent receipts on the basis that those
receipts are conrmation by the customers that the amounts were owed.
The auditor would need to ensure, at least on a test basis, that receipts
were in fact from the customer and related to the matched invoice. The
entire accounts receivable (A/R) population could thus either be
conrmed or identied as still outstanding and therefore worthy of
special investigation as risks. Conrmation directly with the customer
could be focused on the invoices not paid by the test date.
MAKING IT HAPPEN
As indicated earlier, the profession has not realized the full potential of
technology to improve audit effectiveness. There are at least three things
the profession could do to accelerate the adoption of better technologies:
r
Encourage audit research and development
r
Provide guidance to practitioners and update auditing standards
to encourage the adoption of better technologies
r
Encourage and recognize new resource models that bring to bear
the new skills required in today’s world to complement traditional
CPA skills
Encouraging Audit Research and Development
The profession should promote research into how data science and
related IT can improve the quality and effectiveness of auditing. Very
little such research is being done by universities, which have the
capability. Nor is much being done by rms, which mostly do not have
much research capability, but certainly have a great deal of auditing
expertise. The vehicle for promoting such research could be a consortium
100
ESSAY 4: REIMAGINING AUDITING IN A WIRED WORLD
of universities, rms, professional bodies, solutions providers, and
experts in related elds such as articial intelligence, machine learning,
statistics, and big data analytics. If successful, such a consortium could
lead to a owering of useful audit research and the development and
implementation of solutions that signicantly improve audit
effectiveness. Funding, governance, and similar issues would need to be
worked out.
Providing Guidance and Updating Auditing
Standards
An impediment to transformative thinking is that basic auditing
standards were set a long time ago and the need to comply with them
discourages auditors from considering how to do things better by doing
them totally differently; in some cases, available technology-enabled
auditing methods would appear to contravene auditing standards
(Titera, 2013). Furthermore, there is virtually no professional auditing
guidance on the theory and practice of applying new data analytic,
continuous auditing, and other techniques and technologies to auditing.
For example, auditing standards recognize audit sampling but there is
nothing that explains or encourages the types of 100 percent tests and
detailed data analyses of entire populations that can signicantly increase
effectiveness. Auditing standard setters should review current standards
and guidance with a view to removing barriers and encouraging the
optimal use of technology to improve audit effectiveness.
Encouraging and Recognizing New Resource
Models
CPAs are required to lead teams’ auditing nancial statements because
accounting is the indispensable eld of knowledge required to perform
an audit of nancial statements and opine whether they are in accordance
with generally accepted accounting principles. However, audits with any
degree of complexity usually require the participation of specialists in
tax, information technology, valuations, statistics, actuarial science, or
other elds, who are not necessarily CPAs. As auditors make increasing
use of the technologies described in this essay, they will be obliged to
depend even more on professionals who have the skills
traditionally-trained auditors lack. In some cases, it will make sense to
have these resources within audit rms. In other cases, it will not.
Regardless, rms will need to reassess their human resources models and
alternative sources to ensure that they strike the right balance.
In the auditing profession as a whole, where there are many auditing
rms and tens of thousands of audits, it seems that there should be
101
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
opportunities for solution providers to offer auditing applications and
skilled resources as a profession-wide service. The advent of cloud
computing creates opportunities for such solution providers to offer
services that do not require software installation and maintenance. There
are clearly condentiality, privacy, and independence challenges that
would need to be overcome, and the profession should take the lead in
doing so.
BLUE SKY SCENARIO REVISITED
Sally is now close to retirement and ponders how AART was replaced by
Eco-AART that changed not only her assurance role but also the business
world where highly automated corporate and audit systems coexist, and
regulations are formalized into software and updated in an ongoing
manner. The audit is now conducted substantially via automated
mechanisms such that an evergreen opinion (AICPA 1999) is dynamically
maintained, multiple audit opinions exist continuously for different
stakeholders, and a pink system status implies the need for immediate
corrective action by members of the audit team. In fact, the audit function
is now one of the most expensive components of the business process, as
it is not fully automated like many of the other robotic processes at MLE.
Furthermore, much of the highly technical competencies are being
provided by specialized staff with hybridized employment links to the
rm. Sally recognizes that the consistent leveraging of advanced
technologies and processes was a key ingredient in the long-term
prosperity of MLE. As she prepares for dematerialization and subsequent
beaming to Jupiter for a tour of the Galilean moons, she reects back to
2013, and barely remembers why so many practitioners resisted the
paradigm shift in auditing for so many years. How times have changed!
REFERENCES
AICPA, 2012. "The World in 2025: Technological Trends." Retrieved on April 21,
2013. www.aicpa.org/research/cpahorizons2025/globalforces/technological/
pages/default.aspx.
AICPA, 2012. "Consideration of Fraud in a Financial Statement Audit," AU-C
Section 240, December, 2012. www.aicpa.org/Research/Standards/
AuditAttest/DownloadableDocuments/AU-C-00240.pdf.
AICPA, 2012. "Understanding the Entity and Its Environment and Assessing the
Risks of Material Misstatement," AU-C Section 315, December, 2012.
www.aicpa.org/Research/Standards/AuditAttest/DownloadableDocuments/
AU-C-00315.pdf.
AICPA, 2012. "Analytical Procedures," AU-C Section 520, December, 2012.
www.aicpa.org/Research/Standards/AuditAttest/DownloadableDocuments/
AU-C-00520.pdf.
102
ESSAY 4: REIMAGINING AUDITING IN A WIRED WORLD
Behrens, J.T., 1997. "Principles and Procedures of Exploratory Data Analysis."
Psychological Methods. Vol. 2, No. 2, 131–160.
Boomer Consulting, Inc., 2012. "Guide to Electronic Conrmations, A Boomer
Advantage Guide." Retrieved on March 20, 2013.
www.slashdocs.com/wqwr/guide-to-electronic-conrmations.html.
CICA/AICPA Study Group, 1999. Research Report: Continuous Auditing. Toronto,
Canada: The Canadian Institute of Chartered Accountants, American Institute
of Certied Public Accountants, 1999.
Financial Times, 2014. SEC, with the Program: The U.S. Regulator Has Invested in
High-Tech Tools to Police Wall Street. May 9, 2014.
Hoogduin, L., Yoon, K., and Zhang, L., 2014. "Integrating Different Forms of Data
for Audit Evidence: Markets Research Becoming Relevant to Assurance,"
Working Paper, CARLab, Rutgers Business School, 2014.
Issa, H., 2013. "Exceptional Exceptions," Ph.D. Dissertation, Rutgers University,
Rutgers Business School, 2013.
Mayer-Sch
¨
oenberger, V., and Cukier, K., 2013. Big Data: A Revolution That Will
Transform How We Live, Work, and Think, Eamon Dolan/Houghton-Mifin
Harcourt, 2013.
Titera, W.R., 2013. "Updating Audit Standards—Enabling Audit Data Analysis,"
Journal of Information Systems, 27, 1, 325–331, Spring, 2013.
Tukey, J.W., 1977. Exploratory Data Analysis, Reading, MA: Addison-Wesley.
Vasarhelyi, M. A., and Srivastava, R., 2002. "Conrmatory Extranets: Rebalancing
the Entire Structure of Audit Evidence," Working Paper, Rutgers, The State
University of New Jersey, Newark.
103
ESSAY 5
Data Analytics for
Financial Statement
Audits
Trevor R. Stewart, CA, PhD
ABSTRACT
Data science and supporting technologies have advanced enormously in
recent years, incorporating theories, techniques, and technologies from
many elds, including mathematics and statistics; computer science;
machine learning, including pattern recognition; data visualization; and
data, text, and process mining. Data analytics (DA) has the potential to
transform the way nancial statement audits are conducted making them
signicantly more effective and possibly more efcient. There is an
increasing recognition of this potential in the profession though few if
any transformative applications have yet emerged, and there is a chronic
shortage of data scientists and very few who understand auditing. There
is an opportunity for rms, universities, professional bodies, standards
setters, regulators, and solutions providers to collectively bring about
transformative change.
THE AUDIT CONTEXT
DA as applied to nancial statement auditing is the art and science of
discovering and analyzing patterns, identifying anomalies, and
extracting other useful information in data underlying or related to the
subject matter of an audit through analysis, modeling, and visualization
for the purpose of planning or performing the audit. DA includes
methodologies for
r
identifying and analyzing anomalous patterns and outliers in data;
105
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
r
mapping and visualizing nancial performance and other data
across operating units, systems, products, or other dimensions for
the purpose of focusing the audit on risks;
r
building statistical or other predictive models that explain the data
in relation to other factors and identify signicant uctuations
from the model; and
r
combining information from disparate analyses and data sources
for the purpose of gaining additional insights.
DA and Generally Accepted Auditing
Standards
DA as just dened is and always has been integral to nancial statement
audits, pre-dating modern methods and technology. Indeed, it is difcult
to imagine an auditor not using DA and still achieving the overall
objectives set forth in auditing standards, which, to recap, are to
a. obtain reasonable assurance about whether the nancial
statements as a whole are free from material misstatement,
whether due to fraud or error, thereby enabling the auditor to
express an opinion on whether the nancial statements are
presented fairly, in all material respects, in accordance with an
applicable nancial reporting framework; and
b. report on the nancial statements, and communicate as required
by generally accepted auditing standards, in accordance with the
auditor’s ndings.
Auditors have historically scanned nancial statements and analyzed
lists or summaries of transactions and balances; scrutinized journals for
anomalous and unusual entries; compared results by month or operating
unit; compared entities’ operations and results with industry peers and
economic expectations; and considered events that might have affected
the entity to ensure that they have been properly accounted for. It was
ever thus, even in pen and ink days. Over the years, technology—
whether it be Excel, ACL, IDEA, or the Internet—has improved the
mechanics of how these tasks are performed, though the processes
themselves have scarcely changed for decades. The emphasis in
introducing technology to the audit process has been on improving both
effectiveness and efciency. While effectiveness has improved, there has
not been the quantum leap that technology has the potential to enable.
What is different now, in the second decade of the 21st century, is that
extraordinary recent advances in fundamental data science, vast
increases in computer power, and access to astronomical amounts of data
and information have converged to provide an environment ripe for DA
106
ESSAY 5: DATA ANALYTICS FOR FINANCIAL STATEMENT AUDITS
that can and is transforming industries. The time is ripe for
transformative thinking in the profession.
DA can contribute to every phase of the audit
r
to pre-engagement activities such as deciding whether to accept or
continue an engagement;
r
to audit planning;
r
to understanding the entity and its environment and assessing the
risks of material misstatement;
r
to evaluating the design and implementation, and testing the
operating effectiveness of internal controls;
r
to substantive testing, both analytical procedures and tests of
details; and
r
to concluding and reporting.
DA is relevant to and has the potential to signicantly improve audit
procedures throughout the audit. Examples include procedures for the
following:
r
Identifying and assessing fraud risk
r
Performing external conrmation procedures, especially the
identication of high risk items for conrmation
r
Auditing accounting estimates
r
Obtaining an understanding of related party relationships and
transactions
r
Obtaining evidence about the valuation of investments, the
existence and condition of inventory, as well as the completeness
of litigation, claims, and assessments
r
Identifying material subsequent events
r
Evaluating whether there is substantial doubt about the entity’s
ability to continue as a going concern
DA techniques provide leverage by analyzing data and presenting results
so that the auditor can more easily make judgments. DA handles the
mechanics while real cognition happens in the mind of the auditor. That’s
as things stand today. But DA is not static and, as described later, recent
developments have led to a new breed of computers that are capable of
higher-order cognitive processes that until now have been the exclusive
preserve of humans. Cognitive computers are destined for an important
role in the accounting and auditing profession.
The auditor’s overall objective of obtaining reasonable assurance can also
be expressed as reducing audit risk, the complement of assurance, to an
acceptable level. Auditing standards explain that audit risk is a function
107
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
of the risks of material misstatement and detection risk; and that the risks
of material misstatement consist, in turn, of two components, inherent
risk and control risk, which are the entity’s risks and exist independently
of the audit of the nancial statements. DA can be used to identify and
assess the risks of material misstatement—both those inherent to the
entity and arising from internal control weaknesses—and can be used to
reduce detection risk by obtaining or producing substantive evidence.
AUDIT APPLICATIONS OF DA
This section describes types of applications of nancial statement audit
DA. Two illustrative worked examples with data are included later in the
essay.
Understanding the Entity, and Risk
Assessment
DA can play a signicant role in helping the auditor understand the
entity and its environment, and identify and assess risks of material
misstatement. Visualization tools and other techniques can help the
auditor understand the business, identify anomalous patterns or outliers,
and ultimately plan the audit.
In an illustrative example described later, DA is used to understand how
the entity compares with its peers across multiple key nancial ratios.
Frequency distributions are constructed of those distributions such that
the entity’s ratios can be located relative to those of its peers. Location in
a tail of the distribution might be an indicator of strength or risk
depending on the ratio and the tail. Locus can be tracked over time as an
indicator of improvement or deterioration.
In contrast, it is sometimes useful to take an unstructured approach
where the auditor does not start with a pre-specied characteristic, such
as nancial ratios, but rather approaches the data in an unstructured
more open-ended way to discover whether there are natural groupings
and, if so, what the determining factors are and whether they have any
potential audit signicance. Cluster analysis is a common technique used
for this type of exercise (Provost and Fawcett 2013). For example, a
cluster analysis might be applied to a bank with hundreds of branches to
discover whether there are natural groupings. The idea is to nd clusters
where branches within the cluster are similar but signicantly different
from branches in other clusters. It is also of interest to detect outliers
within clusters—that is, branches near the fringe of the cluster to which
they belong.
108
ESSAY 5: DATA ANALYTICS FOR FINANCIAL STATEMENT AUDITS
Performing Substantive Analytical Procedures
Analytical procedures consist of evaluations of nancial information
through analysis of plausible relationships among both nancial and
nonnancial data. Scanning is a type of analytical procedure involving
the auditor’s exercise of professional judgment to review accounting data
to identify signicant or unusual items to test.
DA techniques may help with scanning by, for example, suggesting
hypotheses about the relationship between data variables. Regression or
visualization software might reveal for a chain of retail stores that sales is
strongly correlated with oor area and ZIP code median income.
However, in order to use the regression model to obtain substantive audit
evidence about sales, the auditor should use professional judgment and
knowledge of the business to be satised that the apparent relationship is
in fact plausible and the regression parameters reasonable. If the auditor
is satised with the model and recorded sales fall within a reasonable
threshold of predicted values, then the auditor will have obtained a
degree of substantive audit evidence and may reduce or eliminate further
substantive audit procedures. The auditor should investigate the cause
and perform additional substantive procedures as necessary for branches
with sales outside the acceptable threshold.
Predictive models are represented by one or more equations linking the
target variable of audit interest (sometimes known as the dependent
variable) to predictor variables (sometimes known as independent
variables). Proof in total, a form of substantive analytical procedure, is one
type of predictive modeling. For example, the auditor might test the
reasonableness of wages via a model such as the following:
PredictedWages(Year 2) = ActualWages(Year 1) × (1 + InationRate)
# Employees(Year 2)
# Employees(Year 1)
Whether this model is reasonable or not is a matter for the auditor’s
professional judgment. If the mix of employees is about the same in both
years and the rate of ination a reasonable proxy for the rate of increase
in wages, then it would be expected that the model prediction will not
differ by much from recorded total wages for the current year. If the mix
of employees has changed or different rates of increase apply to different
categories of employee then the model could be extended. Similarly,
published interest rates and average balances may be input to a
multiplicative model to predict total interest and compare it with the
recorded amount; and for some fungible commodities total sales can be
predicted if there is reliable data on quantities and prices. In some cases,
109
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
a proof in total may be all that is required to audit an account. Of course
the validity of the test depends on the reliability of the data used and the
expected accuracy of the formula. The model could be more complex
than a simple equation and it could incorporate a stochastic element to
account for expected random variation. Such applications may also be
used in conjunction with population analysis.
In statistical predictive modeling, the statistical behavior of "training
data," typically audited data from prior periods, is analyzed to identify
and parameterize variables in a model and that model is then used to
predict using current data. For example, in auditing revenues a
regression analysis may be performed on data for 36 prior months for
sales and cost of goods sold to establish a model and determine its
standard error and other statistics. Provided that there are grounds for
believing that the same model should apply in the current period, it may
be used by plugging monthly cost of goods sold into the equation to
predict sales and determining whether residuals (recorded minus
predicted values) are signicant. Such applications are typically used for
substantive analytical procedures.
Analyzing and Testing Populations of Detailed
Transactions and Balances
Performing detailed audit tests on a small sample of items was necessary
when items had to be selected from printed or hand-written listings and
computations had to be performed manually. With computerized data
and le interrogation audit software, however, many tests can be
performed on 100 percent of the population. It is also possible to
simultaneously analyze and visualize the complete population in ways
that can reveal unexpected patterns and outliers worthy of special
investigation. Sampling is still necessary for certain procedures—for
example, the physical inspection or third-party conrmation of assets, or
the analysis of complex contracts. Nevertheless, other procedures may be
performed on the entire population, thus increasing audit assurance at
little additional cost. Many characteristics that might have required
human inspection in the past can be performed automatically—for
example, determining whether the vendor or customer is approved, or
whether the sale or expenditure is customary. A complete population
analysis can also be used to stratify a population so that riskier items can
receive increased audit focus. Where the items are complex legal
contracts, text analysis might be used to ag potentially problematic
clauses across the entire population in order to focus the auditor’s
inspection.
110
ESSAY 5: DATA ANALYTICS FOR FINANCIAL STATEMENT AUDITS
An external conrmation represents audit evidence obtained by the auditor
as a direct written response to the auditor from a third party (the
conrming party) in paper form or by electronic or other medium. DA
software can be used to comprehensively analyze entire populations to
identify and focus the conrmation sample on those that are most at risk.
For accounts receivable, outstanding invoices at conrmation date may
be matched up with payments received subsequently, thus reducing the
population to be sampled by eliminating items known to have been
paid.
1
For accounts payable, vendors with whom the company does a
high volume of business can be selected for conrmation. Bank
conrmations can be largely automated using a combination of DA
software and a third-party conrmation service.
Recalculation consists of checking the mathematical accuracy of
documents or records. Analytical software can often perform
recalculations on 100 percent of a population, and ensure also that correct
master le data have been used. This is an example of where the evidence
that is possible with analytical software is far superior to that obtained
when recalculation is limited to a sample.
Considering and Testing for Fraud
Because of the risk that management could override controls and post
fraudulent journal entries, auditors are required to test the
appropriateness of journal entries recorded in the general ledger and
other adjustments made in the preparation of the nancial statements,
including entries posted directly to nancial statement drafts.
There are numerous journal entry-testing software systems used in
practice, developed either by individual rms or commercial solutions
providers. Because of the volume of journal entries processed by today’s
ERP systems, many of the testing systems in use today produce too many
false positives, leaving it up to the auditor to gure out which "positives"
are genuinely problematic and how to deal with the rest. There is a need
for software that does a better job of ltering—just as credit card
companies tend to do a good job of alerting customers to potential fraud
without overwhelming them with false alerts.
Collusive fraud can be particularly difcult to detect. One detective
technique, used in forensic work, is to analyze social networks via
software that maps networks of people who interact. This type of
analysis can be used to look for related parties and collusive fraud, and to
analyze and evaluate separation of duties for internal control purposes.
1
The validity of this depends on the accuracy and reliability of the subsequent receipts data and
on ensuring that receipts actually relate to the balances being tested.
111
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Testing the Operating Effectiveness of
Internal Control
Reperformance involves the independent execution of procedures or
controls that were originally performed as part of the entity’s internal
control.
Internal control procedures such as ensuring that prices on an invoice
come from an approved price list, performing account reconciliations, or
ensuring via batch totals that information is correctly transferred from
one system to another can often be reperformed on a 100 percent basis
rather than for a sample and can be performed continuously.
Observation consists of looking at a process or procedure being performed
by others (for example, the auditor’s observation of inventory counting
by the entity’s personnel or the performance of control activities). As
high-performing computer systems take over processes and procedures
that previously required human involvement, the nature of audit
observation changes. Further, when a computer performs a control
activity, it can be monitored around the clock and any lapses immediately
reported.
Process mining of event logs is a technique that can identify internal control
deciencies such as payments made without approval, violations of
segregation of duty controls, and violations of entity-specic internal
procedures (Jans, et al. 2014).
Inquiry
Inquiry consists of seeking information of knowledgeable persons, both
nancial and nonnancial, within the entity or outside the entity.
Statistical analysis, relationship analysis, and the summarized results lists
of anomalies provided by population analysis can all provide signicant
material for client discussions (and often provide insights that clients nd
valuable).
ALOOK AHEAD:COGNITIVE COMPUTING IN
THE
AGE OF BIG DATA
Utilizing Big Data
Big data is the product of a technological environment in which almost
anything can be recorded, measured, and captured digitally, and thereby
turned into data. Big data is important in machine learning, where
112
ESSAY 5: DATA ANALYTICS FOR FINANCIAL STATEMENT AUDITS
computers are trained in a subject by ingesting a vast number of
examples and other information. As the next section will explain, it’s a
key to cognitive computing. Essay 4, "Reimagining Auditing in a Wired
World," discusses big data in more detail.
Cognitive Computing
In 2011, IBM’s Watson computing system became the world’s best
Jeopardy! player. Jeopardy!, a TV quiz show, is known for its complex,
tricky questions and very smart human champions. Playing it requires
not only acquiring the general knowledge needed by humans to play but
the ability to answer questions posed in nuanced natural language,
including puns, synonyms and homonyms, slang, and jargon (Friedman
2014)—something that humans are very good at and computers have
historically been very bad at. The same cognitive technology is being
applied to medicine and other elds, and some think that "Dr. Watson"
may soon become the world’s best diagnostician. Google’s self-driving
car is another example of how cognitive computers can learn to perform
"human" tasks with relentlessly super-human skill.
Enabling all this are recently developed deep learning algorithms that
augment and enhance AI algorithms developed over decades since the
1950s; big data, which provide the raw material that learning algorithms
ingest and "understand" as the basis for their knowledge; and
inexpensive graphical processing chips originally designed for video
games but repurposed for machine learning via clever new techniques
(Kelly 2014). Useful AI—once an oxymoron—is now a reality.
Cognitive computers combine AI and machine-learning algorithms, in an
approach that attempts to reproduce the behavior of the human brain
(Mohda 2014, 28-29). Instead of being programmed in a traditional way,
cognitive computers learn, as do humans, by seeing many instances of
what they are learning about. Just as a child learns to distinguish
between dogs and cats, a cognitive computer is trained by being fed
countless examples from a world of big data together with human
intervention where required ("No, that’s not a dog. It’s a kitty.").
Dr. Watson has learned medicine by "reading" dozens of textbooks and
medical journals, and thousands of patient records from Memorial Sloan
Kettering (Friedman 2014). Writing for Forbes Bruce Upbin (2013)
reported, "Watson has analyzed 605,000 pieces of medical evidence, 2
million pages of text, 25,000 training cases and had the assist of 14,700
clinician hours ne-tuning its decision accuracy." And of course Dr.
Watson continues to learn, to keep up to date, and to improve: more CPE
than would be possible for any human doctor.
According to the Financial Times (Alloway and Massoudi 2014), Kensho is
an analytics platform being designed to instantly answer millions of
113
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
complex nancial questions by automating previously human-intensive
research, for example, "What happens to US homebuilder stocks if a
category three hurricane makes landfall?" Goldman Sachs and Google are
investors. Most of the data that move markets are inherently
unstructured—central bank announcements; geopolitical developments;
product releases; research breakthroughs; droughts, hurricanes, and
other weather-related phenomena; and natural disasters. To advise its
human handler, a cognitive computer, like a Watson or a Kensho must
apply heuristics learned from analyzing and making sense of vast
amounts of mostly unstructured textual data plus a (usually much
smaller) corpus of structured quantitative data.
If cognitive computers can be trained to be superb medical or nancial
assistants there is no reason they could not be superb CPA assistants. The
cognitive CPA could ingest and "understand" all available accounting
and auditing knowledge that exists in the form of professional standards,
interpretations, guidance, journal articles, and other literature; and SEC
and other regulatory rules, rulings, pronouncements, and millions of
lings. No human CPA could possibly absorb, retain, and constantly
update this amount of information. It is no longer unreasonable to
predict that before too long a computer will be able to review a set of
nancial statements, including disclosures and management’s discussion
and analysis, and identify problems as effectively as the most
experienced CPAs. Unlike humans, the cognitive CPA would be
consistent—always giving the same answer to the same question (unless
its knowledge has been updated)—and would never be grumpy or
sleep-deprived. While expensive to develop and train, the cognitive
CPA’s marginal cost per enquiry would be essentially zero; and the
cognitive CPA would operate in the cloud and be available to any user
with a smartphone regardless of time or location.
UPPING OUR GAME
The essay "Reimagining Auditing in a Wired World" suggests steps the
profession should take to realize the full potential of technology to
improve audit effectiveness. Those steps are especially important if DA is
to be used to transform the way audits are performed. In particular,
normative research is required into how DA can be integrated into the
audit so that it replaces current procedures where appropriate and results
in signicantly greater effectiveness, not simply an incremental
improvement. Specic methodologies and techniques need to be
developed that can be rolled out widely and effectively implemented in
the eld. There are also many practical issues that require research such
as determining the proper role of DA in risk assessment and internal
control evaluation; how to ensure high quality results by focusing on
114
ESSAY 5: DATA ANALYTICS FOR FINANCIAL STATEMENT AUDITS
data quality, and documentation and review issues; exploring the
assurance implications of 100 percent population tests as well as practical
impediments such as dealing with false positives; reliance on internal
auditor’s use of DA; how to interpret DA ndings; and the efciency and
effectiveness consequences of DA (Wang and Cuthbertson 2015).
ILLUSTRATIVE EXAMPLES
This section includes two examples of audit DA. The rst shows how the
use of some simple graphics can enhance understanding.
2
The second
illustrates the use of peer data.
Example 1: Simple DA Visualization
The AICPA Audit Guide Analytical Procedures (2012) illustrates the use of
analytical procedures in both planning and substantive testing using a
case study for a chain of convenience stores, called On the Go Stores.Inthe
case study, trend analysis, ratio analysis, reasonableness tests, and
regression analysis are demonstrated. Here we will use the case study
data to illustrate the use of graphics in DA as a way to better understand
the entity’s business and identify risks of material misstatement. On the
Go Stores has 23 convenience stores located in the Southeast United
States. The data are shown in table 5-1.
Table 5-1: On the Go Stores, Case Study Data
Store
Prior-Year
Sales
(Audited) ($)
Current-
Year Sales
($)
Current-
Year
Inventory
($)
Square
Feet
Average
Number
Full-Time
Employees
Sells
Gas (1)
1 781,793 48,725 2,500 11.00 0
2 1,165,221 1,146,438 44,171 2,500 11.31 0
3 1,147,430 1,195,004 45,714 2,500 12.46 0
4 951,784 37,218 4,000 11.86 0
5 2,037,463 1,981,409 45,826 4,000 10.06 1
6 2,257,920 2,300,671 53,862 4,000 11.10 1
7 1,850,354 1,956,481 49,883 4,000 10.71 1
8 1,916,884 1,799,713 47,016 4,000 7.50 1
9 1,833,209 1,820,641 59,726 4,000 14.00 0
10 774,954 35,882 2,500 11.20 0
11 980,484 1,159,004 37,664 2,500 11.60 0
2
The charts in this section were all prepared in Excel.
115
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Table 5-1: On the Go Stores, Case Study Data—continued
Store
Prior-Year
Sales
(Audited) ($)
Current-
Year Sales
($)
Current-
Year
Inventory
($)
Square
Feet
Average
Number
Full-Time
Employees
Sells
Gas (1)
12 1,069,652 1,139,475 34,662 2,500 12.70 0
13 948,522 44,782 4,000 11.86 0
14 1,795,123 1,984,777 38,774 4,000 12.20 1
15 2,119,015 2,293,847 55,423 4,000 11.10 1
16 1,947,303 1,984,722 52,884 4,000 10.40 1
17 1,705,789 1,798,336 46,834 4,000 8.84 1
18 2,396,971 2,484,503 53,772 4,000 12.10 1
19 1,901,631 1,837,400 43,982 4,000 9.70 1
20 1,514,798 1,609,385 44,893 4,000 7.20 1
21 1,886,587 1,874,229 37,665 4,000 10.50 1
22 698,333 33,826 2,500 10.50 0
23 1,092,908 1,198,229 44,857 2,500 10.90 0
Total 30,618,742 35,719,650 1,038,041 80,000 250.80 12
Source: AICPA, Audit Guide: Analytical Procedures, March 2012
Five of the 23 stores (stores 1, 4, 10, 13, and 22) opened during the year.
Operations vary by geographic location and the mix of products sold. The
location of a store is based on several factors, such as competition and the
economic environment of the location. Typically, a store’s operations do
not change much unless a new product line is introduced, such as selling
gas, offering check-cashing services, or selling lottery tickets. The mix of
products and services can vary, and the most important factor is whether
the store sells gasoline. (Stores 5, 6, 7, 8, 14, 15, 16, 17, 18, 19, 20, and 21
sell gasoline.) These additional product lines typically affect the volume
of customers as well as the number of full time employees.
One could stare at table 5-1 for some time without easily seeing patterns
and correlations. For most people, data comes to life when it is presented
graphically. One could start by simply charting sales for the two years as
it is presented in the table. Figure 5-1 does this as a dot plot, with store ID
on the horizontal category axis, sales on the vertical value axis, with
current sales indicated by an empty circle and prior sales by a solid circle.
First, it is interesting that the stores appear to comprise several clusters:
{#1–#4}, {#5–#9}, {#10–#13}, {#14–#21},and{#22–#23}. The apparent
clustering may have to do with location or some other factor correlated
with store ID, or it could just be a spurious pattern thrown up by
coincidence. It might be worth nding out.
116
ESSAY 5: DATA ANALYTICS FOR FINANCIAL STATEMENT AUDITS
Figure 5-1: On the Go Stores, Current and Prior Year Sales in Store ID
Order as Presented
Store Sales, Current and Prior Years
Figure 5-1 can be changed to reveal new information in at least two ways.
First, rather than plotting the numbers in store ID order, plotting them in
order of store sales shows the distribution of the stores from lowest sales
to highest sales. Second, because we know that it makes a signicant
difference whether or not the store sells gas, it makes sense to distinguish
between those that sell gas and those that don’t. The result is in gure 5-2.
Certain things are immediately apparent, for example:
r
The ve new stores that opened in the current year have the lowest
sales, conrming something one might expect.
r
Stores that sell gas have signicantly higher sales than those that
don’t. The one exception is Store #9, which appears to be
performing as well as the stores that sell gas. The auditor might
want to check whether the store has been correctly classied as a
no-gas store, and, if it has, enquire as to why it appears to have
done so much better than the others.
The case study discusses reviewing sales per square foot compared to the
benchmark amount of $490 provided by National Association of
Convenience Stores (NACS) as a reasonableness test. The results are
plotted in gure 5-3. Once again the ve recently opened stores have the
lowest sales per square foot. Stores that don’t sell gas are all performing
at below the industry benchmark; three of those that do sell gas are
operating well above the benchmark.
Figure 5-4 hows that inventory turnover (Sales ÷ Inventory) is
signicantly greater for stores selling gas (36 to 50 times) than for stores
not selling gas (16 to 33 times).
117
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 5-2: On the Go Stores, Current and Prior Year Sales in Store Sales
Order Showing Stores Selling and Not Selling Gas as Separate Series
Store Sales, Current and Prior Years, in Sales Order
Figure 5-3: On the Go Stores, Sales per Square Foot by Store Compared
with NACS Benchmark
Sales per Square Foot Compared with NACS Benchmark ($490)
Figure 5-4: On the Go Stores, Inventory Turnover by Store
Inventory Turnover
118
ESSAY 5: DATA ANALYTICS FOR FINANCIAL STATEMENT AUDITS
Figure 5-5 is a scatter plot of sales against number of employees, a
different type of chart from the dot plots depicted, which all have
horizontal category (store ID) rather than value axes. When the data are
broken out as two series depending on whether or not the store sells gas
there is a strong correlation, approximately 80 percent in each case, and it
is clear that stores selling gas have signicantly higher sales per
employee than those not selling gas. If the data were not broken out into
two series, the regression line would not be a good t and the correlation
would be only 15 percent. It would be inappropriate for the auditor to
develop a single regression model for the data as a whole without
allowing for the difference between the two sets of stores. This serves to
make the point that auditors should use DA to understand the data
before attempting to model it.
Figure 5-5: On the Go Stores, Scatterplot of Sales Versus Average
Number of Employees
Correlation between Sales and Number of Employees
Source: Compustat 2013
The conclusions and insights that the auditor of On the Go Stores would
draw from this DA would depend on the specics of the entity, including
the auditor’s expectations. The visualizations help the auditor see
patterns and relationships and possibly unexpected results. It is up to the
auditor to decide, based on other knowledge of the business, what is
important and what, if anything, requires additional audit focus.
119
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Example 2: Financial Ratio Peer Analysis
3
An important indicator of nancial health or lack thereof and thus of
interest to an auditor trying to understand the entity and assess risk is
how the entity’s key nancial ratios compare with those of its industry
peers. In this example, we show frequency distributions of several key
ratios for the wholesale and retail sector (SIC codes 5000 to 5999) against
which the ratios of any audit client in that sector can be compared. If a
particular ratio is located in the tail end of the distribution it indicates a
deviation from the industry norm, which may be good or bad depending
on the ratio, which tail, and how far out. A visualization of the complete
distribution allows the auditor to make a judgment based on a complete
picture of the sector; the client ratios can all be viewed juxtaposed against
the relevant distributions and viewed together dashboard-style; and the
ratios and their relative positions tracked year to year for signs of
improvement or deterioration.
The Compustat database, which includes approximately 430 companies
in the SIC 5000-5999 sector, was used to develop the distributions from
2013 data. For example, gure 5-6 shows the distribution of the current
ratio (current assets divided by current liabilities). It can be seen that the
distribution is very highly skewed. Just looking at the distribution it
would seem that current ratios between 1 and 3 are the norm, ratios less
than 1 are potentially worrisome, and a very high ratio (the highest in the
database is nearly 16) might indicate a potential problem with the data,
possibly a misclassication of current assets or current liabilities.
While it is helpful to visualize the shape of the distribution, cumulative
frequency distributions are generally more useful because percentiles can
be read directly. Cumulative frequency distributions for several ratios
derived from the Compustat database are displayed in gure 5-7. The
rst chart is the cumulative distribution for the current ratio and
corresponds to the frequency distribution in gure 5-6. Additional ratios
presented include inventory turnover, return on assets (ROA), and
long-term debt to equity. A table of statistics is included to the right of
each chart. It can be seen from the cumulative distribution, for example,
that the median current ratio, the 50th percentile, is approximately 1.7.
It’s the point on the horizontal axis corresponding to 50 percent on the
vertical axis. The table gives it more accurately as 1.657. It can be seen
that only 5 percent of this peer group have current ratios of 0.740 or less,
so the auditor who considers a low ratio to be a risk might have concerns
if the entity’s ratio is that low.
3
I thank Paul E. Byrnes of Rutgers Business School for providing me with an unpublished paper
on which the examples and data in this section are based.
120
ESSAY 5: DATA ANALYTICS FOR FINANCIAL STATEMENT AUDITS
The conclusions and insights that the auditor of an entity in the
wholesale and retail sector would draw from this DA would depend on
the specics of the entity, including the auditor’s expectations. The
visualizations help the auditor locate the entity’s ratios relative to the
entire universe of peers, but it is up to the auditor to decide, based on
other knowledge of the business, what is important and what, if
anything, requires additional audit focus. This is an example of DA
applied to an industry segment, the results of which could be leveraged
across multiple audits in that segment.
Figure 5-6: Current Ratios in the Wholesale and Retail Sector
Current Ratio Frequency Distribution
Source: Compustat 2013
121
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 5-7: Cumulative Frequency Distributions for Various Ratios
122
ESSAY 5: DATA ANALYTICS FOR FINANCIAL STATEMENT AUDITS
Figure 5-7: Cumulative Frequency Distributions for Various
Ratios—continued
123
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure 5-7: Cumulative Frequency Distributions for Various
Ratios—continued
124
ESSAY 5: DATA ANALYTICS FOR FINANCIAL STATEMENT AUDITS
Figure 5-7: Cumulative Frequency Distributions for Various
Ratios—continued
Multivariate Ratio Analysis
The charts in gure 5-7 allow individual ratios to be benchmarked. While
such a piecemeal technique has value, it may fail to identify relationships
exhibited via the combination or synthesis of all examined ratios. Because
outliers in univariate space are often not found to be multivariate outliers
(Starkweather 2013), if one is looking for true outliers, it may be useful to
benchmark several key ratios simultaneously. Thus, instead of reviewing
Ratio
1
, Ratio
2
, ..., and Ratio
n
, separately, one might consider the n-tuple
(Ratio
1
, Ratio
2
, ..., Ratio
n
) as a point in multidimensional space to be
compared with the industry benchmark point of median ratios.
125
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
If there are only two ratios, say current ratio and ROA, a scatter plot
shows how the point pairs (Current Ratio, ROA) are dispersed. Figure 5-8
illustrates this using the Compustat-derived data for these two ratios
summarized in gure 5-7. For comparability, scales have been changed so
that 0 is the lowest for each ratio and 1 is the highest, with the relative
measurement retained. Outliers are those points that lie a signicant
distance from the median. In gure 5-8, X marks the median point and
the farthest outlier is circled. To use this chart in practice the auditor
would compute the entity’s current ratio and ROA and compare the pair
to the points in gure 5-8, and would typically be concerned if the point
is in the "weak" lower left. gure 5-9 shows the distribution of the
distances of the points depicted in gure 5-8 from the median point
marked with an X.
If there were three ratios, the scatter of points (Ratio
1
, Ratio
2
, Ratio
3
)
would occupy a three-dimensional "box" chart rather than the at sheet
occupied by gure 5-8. For more than three dimensions we lose the
ability to depict the points graphically though they can be handled just as
easily mathematically. The equivalent of gure 5-9 works regardless of
the number of dimensions.
Figure 5-8: Dispersion of the Ratio Pair (Current Ratio, ROA)
126
ESSAY 5: DATA ANALYTICS FOR FINANCIAL STATEMENT AUDITS
Figure 5-9: Distribution of (Current Ratio, ROA) Distances from the
Median Point
Distribution of (Current Ratio, ROA) Distances from Median
Where multivariate ratio analysis is used, the DA software would
compute distance of the entity’s multivariate ratio point from the median
point and the auditor would compare that distance with the distribution
of distances (for example, gure 5-9) to determine the extent to which the
entity is an outlier. Distance as measured in this multivariate example
accords with our everyday notion of distance—the length of the straight
line joining two points. Data science also recognizes other distance
metrics as well (Starkweather 2013; Tan, et al. 2006). Because ensembles
of multiple approaches have been shown to be particularly effective in
identifying true outliers (Zimek et al. 2014), DA applications seeking true
outliers in a multivariate ratio analysis might compute distance from the
median in several different ways and then calculate a composite outlier
score. For example, if four different measures are calculated, they can be
normalized to lie between 0 and 1 so that the composite outlier score lies
between 0 and 4, with 0 indicating that the point coincides exactly with
the median and 4 indicating that the point is a maximal outlier. Scores
might also be weighted in the construction of the composite.
The literature on multivariate ratio analysis in auditing is thin to
non-existent (Google hits: zero) and the discussion in the preceding
paragraph is speculative. The distance measures of data science are
principally used in applications such as text analysis, and the extent to
127
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
which they would be useful in multivariate ratio analysis in auditing and
how one would interpret results is an open question that would benet
from theoretical and applied research. Questions such as what ratios in
what industries are of most audit interest and how they can best be used
would also benet from such research.
REFERENCES
Alloway, T., and A. Massoudi. "Goldman Sachs Leads $15m Financing of Data
Service for Investors." Financial Times (November 24, 2014).
Behrens, J. "Principles and Procedures of Exploratory Data Analysis."
Psychological Methods vol. 2, no. 2 (1997): 131–160.
Box, G., and N. Draper. Empirical Model-Building and Response Surfaces.NewYork:
Wiley, 1987.
Friedman, L. "IBM’s Watson Supercomputer May Soon Be the Best Doctor in the
World. Business Insider (April 22, 2014), www.businessinsider.com/ibms-
watson-may-soon-be-the-best-doctor-in-the-world-2014-4.
Jans, M., M. G. Alles, and M. A. Vasarhelyi. "A Field Study on the Use of Process
Mining of Event Logs as an Analytical Procedure in Auditing." The Accounting
Review vol. 89, no. 5 (September 2014): 1751–1773.
Kelly, K. "The Three Breakthroughs that Have Finally Unleashed AI on the
World." Wired (October 2014),
www.wired.com/2014/10/future-of-articial-intelligence/.
Mayer-Sch
¨
oenberger, V., and K. Cukier. Big Data: A Revolution that Will Transform
How We Live, Work, and Think. New York: Eamon Dolan/Houghton-Mifin
Harcourt, 2013.
Modha, D. (interview). "A computer that Thinks." New Scientist (November 2014).
Provost, F. and T. Fawcett. Data Science for Business. Sebastopol, CA: O’Reilly
Media. 2013.
Starkweather, J. "Multivariate Outlier Detection with Mahalanobis Distance."
(2013), www.unt.edu/rss/class/Jon/Benchmarks/Moutlier_JDS_July2013.pdf.
Tan, P., M. Steinbach, and V. Kumar. Introduction to Data Mining. Boston, MA:
Pearson Education, Inc. 2006.
Titera, W. "Updating Audit Standards: Enabling Audit Data Analysis." Journal of
Information Systems, 27, 1, 325–331, Spring, 2013. 2013.
Tukey, J. Exploratory Data Analysis. Reading, MA: Addison-Wesley. 1977.
Upbin, B. "IBM’s Watson Gets Its First Piece of Business in Healthcare." Forbes
(February 8, 2013), www.forbes.com/sites/bruceupbin/2013/02/08/ibms-
watson-gets-its-rst-piece-of-business-in-healthcare/.
Wang, T. and R. Cuthbertson, "Eight Issues on Audit Data Analytics We Would
Like Researched." Journal of Information Systems,
dx.doi.org/10.2308/isys-509552015.
Zimek, A., R. Campello, and J. Sander. "Ensembles for Unsupervised Outlier
Detection: Challenges and Research Questions." SIGKDD Explorations. (2014).
(SIGKDD is the Association for Computing Machinery’s Special Interest Group
on Knowledge Discovery and Data Mining.)
128
ESSAY 6
Managing Risk and the
Audit Process in a
World of Instantaneous
Change
Paul Byrnes, CMA
Gerard Brennan, CFE, PhD
Miklos Vasarhelyi, PhD
Daehyun Moon, CPA
Satyajeet Ghosh, MS, MSE, MBA, CIA, CISA, CFE
ABSTRACT
We live in a world in which business circumstances change and risks
evolve in a very rapid manner. Within this context,
traditionally-developed annual audit plans may not be able to account
for the constantly changing landscape and associated modications to
business risk. Consequently, an approach for regularly monitoring,
assessing, and reporting on all pertinent risk factors should be
implemented. In this setting, auditors would benet by keeping audit
plans current, thus facilitating adjustment of audit procedures and tests
so as to help ensure that a high level of assurance can be provided in a
timely manner.
In this essay, the concept of continuous risk monitoring and assessment
(CRMA) is described as a method for simultaneously conducting effective
audits and managing business operations in the evolving real-time
129
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
global economy. Discussion includes a general overview of CRMA as
well as more specic considerations such as identication of risk,
development and maintenance of key risk indicators, and responses to
changing risk levels by both auditors and management. In summary,
CRMA offers promise in accommodating auditing and business needs of
the future.
INTRODUCTION
In the traditional auditing and managerial environments, risk assessment
is viewed as a discrete activity that occurs periodically at a point in time.
For example, in the external auditing context, risk assessment would
historically occur during the planning stage of the annual audit. The
accumulated evidence would then contribute to deciding pertinent
characteristics of the corresponding audit plan, such as scope,
procedures, and tests, as well as accounts and balances to be emphasized.
However, problems can arise when a client’s risk prole experiences
signicant change between the risk assessment phase and completion of
the associated audit engagement. In this setting, dynamic adjustment of
the audit plan is warranted, but if risk assessment is not a uid activity,
this will not transpire. Problems associated with the untimely
identication of changes in the risk landscape will also affect the
management and internal audit areas.
Increasingly, we live in a society where business circumstances and risk
factors can and do change very abruptly, thus substantially reducing the
lead time for effectively responding to changing conditions.
Simultaneously, stakeholders’ needs and expectations are expanding
(NACD 2009). Within this environment, the traditional method of risk
assessment becomes untenable. Alternatively, a more agile, real-time
approach for monitoring and assessing risks is necessary such that the
audit plan and enterprise risk management protocols are both able to be
updated in real time based upon changing risk levels. Audit and risk
management procedures may then be rened accordingly, thus
improving both audit quality and business productivity. CRMA
demonstrates the potential to facilitate this process.
CRMA ARCHITECTURE—OVERVIEW
Vasarhelyi, Alles, and Williams (2010) initially envisioned CRMA as a
vehicle for monitoring the risk landscape, measuring pertinent indicators,
and providing meaningful input to the audit plan, so that it may be
reformulated dynamically in response to modications in an entity’s risk
130
ESSAY 6: MANAGING RISK AND THE AUDIT PROCESS
prole. The authors also contend that CRMA should ideally be fully
integrated within a structure that includes both Continuous Controls
Monitoring (CCM) and Continuous Data Assurance (CDA) such that a
robust system of continuous auditing is ultimately achieved (gure 6-1).
Figure 6-1: Integrated Components in Continuous Auditing
(Vasarhelyi, Teeter, Krahel 2010)
In practice, there might be an initial tendency to implement one or more
of the modules in a mutually exclusive manner. For instance, an
organization might elect to adopt CRMA as part of a risk management
initiative but not incorporate it with CCM or CDA at that point. While
this will certainly be benecial, eventual integration of all three
components will produce synergistic effects. For example, as the CCM
module identies internal control failures and other issues, this
information can be provided as input to the CRMA program, thus
potentially impacting one or more of the risk measures being monitored,
as well as offering evidence pertaining to whether revision of the risk
prole and corresponding metrics (for example, key risk indicators) is
advisable. Furthermore, as risk measures uctuate in the CRMA system,
this information can serve as input to the CCM module, allowing it to
proactively address changes in business risk (Moon 2014a). While the
CRMA architecture can add value by itself, it can be fully leveraged when
combined with CCM and CDA to provide a complete system of
continuous auditing and monitoring.
131
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
CRMA—GENERAL PROCESS
Vasarhelyi (2011) further describes the CRMA architecture by elaborating
on and discussing its various components and functionalities (gure 6-2).
Figure 6-2: Schemata of CRMA Process (Adapted from Vasarhelyi 2011)
In the proposed CRMA framework, three general types of risks are
identied and monitored. These include business process or operational
risks, environmental risks, and black swans (Taleb 2010). Business
process risks are largely inherent and attributable to the business itself as
well as its industry. Environmental risks include other forces in the
internal environment such as infrastructure and information security
issues, and risks in the macro-environment including those residing in
the political, competitor, and economic arenas (Kuenkaikaew and
Vasarhelyi 2013; Hill 2008). A black swan is a risk that has a very low
probability of transpiring, but would likely carry substantial costs if
materialized. A black swan is formally dened as "an event or occurrence
that deviates beyond what is normally expected of a situation and that
would be extremely difcult to predict." (Financial Times 2014).
Furthermore, the emergence of a black swan can have catastrophic and
unpredictable outcomes. Consequently, although this type of event is
problematic to anticipate, it is vital that monitoring mechanisms are
established to accumulate and maintain information about these
132
ESSAY 6: MANAGING RISK AND THE AUDIT PROCESS
potential risks. For example, in the airline industry, a possible black swan
might be the sudden and prolonged unavailability of jet fuel arising from
shipment sabotage and hijacking as well as ongoing civil unrest.
Obviously, this set of circumstances could have devastating ramications
and, therefore, should be contemplated to the extent feasible.
For each risk identied, a key risk indicator (KRI) or a set of KRIs is
constructed along with an associated benchmark or collection of
benchmarks so that effective monitoring of the risk prole can be
performed in a systematic and continuous manner. As KRI measures (or
combinations of them) are found to be indicative of emerging problems,
vulnerable business process areas and associated accounts and
transactions are identied, the audit plan is updated, and a revised set of
audit procedures and tests is compiled for execution. In addition, the
process facilitates revisions to enterprise risk management procedures. To
complete the CRMA framework, a method for maintaining the set of
KRIs is developed whereby new measures are added, existing metrics are
rened, and stale KRIs are deleted as risk prole modications dictate.
Specic considerations in implementing and using CRMA are explored in
the next section.
CRMA—MORE DETAILED CONSIDERATIONS
Initial construction of the risk prole will be a prerequisite to creation of a
functional CRMA program. In seeking preliminary guidance and
direction, one might initially consult with the Committee of Sponsoring
Organizations (COSO) 2013 Internal Control Integrated Framework. For
an established entity, it will also be benecial to refer to existing
documentation, such as the most recently formulated risk assessment,
risk prole information, or organizational quarterly and annual reports,
and subsequently revise the risk prole as warranted. In achieving this,
the CEB Risk Management Leadership Council (2013) recommends
working closely with all identied risk owners and subject matter
experts. The nalized risk prole will become instrumental in developing
a comprehensive set of KRIs for use in monitoring and assessment
routines.
Risk Identication and Analysis
Scandizzo (2005) outlines a system for managing operational risk and
indicates that mapping is a critical component of the risk identication
and management process. Generally speaking, mapping involves
decomposing and examining a given business process by activity to
identify all pertinent risk drivers (that is, people, processes, systems, and
external dependencies) and associated risk factors. A useful question to
133
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
ask of each activity during this phase is "What can go wrong?" because
this will enhance discovery of relevant task characteristics such as how
an activity can fail, how much risk exists, and the probability of
occurrence and impact of potential risk exposure. Once all risks are
identied and a prole is constructed, efforts gravitate toward the
formulation and implementation of a meaningful set of KRIs able to
productively monitor the risk environment.
KRI Development and Implementation
Key risk indicators are "metrics used by an organization to provide an
early signal of increasing risk exposures in various areas of the
enterprise" (CEB 2013). Each KRI should be a root cause identier or a
leading indicator of risk, such that a proactive response can be made in
addressing detected problems. However, achieving this in practice is a
non-trivial task. In a recent ERM function survey, 56 percent of
respondents indicated the greatest difculty in KRI development is
establishing metrics that are leading indicators of risk (CEB 2013).
Therefore, serious attention must be devoted to KRI construction. For
example, if the inability to settle current liabilities is an identied risk,
several potential KRIs might be envisioned, but many would be
suboptimal. The current ratio might initially be suggested as a potential
KRI in this situation, but at the point when this measure is indicative of a
problem, it could be too late to prevent occurrence of the underlying risk
event. Conversely, customer nancial health trend information is a
leading and more suitable KRI. In this case, if signicant nancial
deterioration in the customer base is detected, then more effective
corrective measures could be enacted to avoid having insufcient
liquidity in clearing current liabilities as they mature.
Moon (2014a) argues each KRI must possess two fundamental
characteristics. First, it has to be measurable, but need not be purely
objective. A KRI can be relatively subjective provided it is independently
quantiable and veriable (Scandizzo 2005). For instance, information
arising via text mining and sentiment analysis of news feeds and other
electronic media could be synthesized in generating an effective KRI. In
an application of this approach, relevant textual information is regularly
analyzed, and variables such as information source quality, frequency
and timing of news, and severity level of content are all determined and
aggregated in computing an entity’s reputation risk score (RepRisk 2014).
On the other hand, for the identied risk of manipulative earnings
management, one might view tone-at-the-top as a potential KRI. The
underlying rationale might be that this variable is highly correlated with
organizational culture as well as the propensity for unethical behavior.
However, evaluating tone-at-the-top would prove extremely challenging
from the standpoint of measurability, and if the proposed KRI is not
134
ESSAY 6: MANAGING RISK AND THE AUDIT PROCESS
objectively quantiable and veriable, it would be an unsatisfactory
indicator.
Second, a KRI must be relevant, meaning that changes in the measure
must result in corresponding alterations in the probability of target risk
event emergence. For instance, employee turnover rate might be
considered as a proxy for risk of having material errors in the nancial
statements. While this seems a reasonable assumption, if the measure is
not found to be highly predictive of the risk event, then it would not be a
useful KRI in this context.
Beyond being measurable and relevant, KRIs should also be
non-redundant, easy to monitor, and auditable (Scandizzo 2005). To meet
the rst of these criteria, if two or more KRIs are highly correlated, then
only one of the metrics is needed. Presumably, the retained KRI would be
that which provides the greatest benet in terms of risk monitoring and
assessment quality. In fullling the second criterion, each KRI should be
relatively easy and cost-effective to measure and report. To meet the nal
requirement, complete documentation of all indicators and
corresponding data sources used for measurement should be consistently
maintained. Table 6-1 provides some theoretical KRIs and associated
applications so as to facilitate initial thinking about risk measure
development.
Table 6-1: Subset of Potential Key Risk Indicators
Key Risk Indicator To monitor risk of problems relative to:
Segregation of duty violations Internal control failures and
misappropriation of assets
Percent of uncollected sales Estimation quality and manipulative
earnings management
Customer nancial health Cash ows, collections, and debt covenants
Customer complaints Sales and customer base; product, labor, and
process quality
Accounting employee turnover Error, fraud, and earnings management
Password reset requests Control failure, fraud, data integrity or loss
Ratio of book value to fair value
for depreciable assets
Estimation quality, errors, and manipulative
earnings management
Customer attrition Revenues and debt obligations
Research and development
spending
Innovation, vision, organizational health,
and management
Tone of media coverage Corporate governance; management
policies and practices
Phishing incidents Controls, external fraud, and data
compromise
135
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
In addition to formulating a comprehensive set of KRIs that are leading,
meaningful measures of associated risk events, benchmarks must also be
incorporated as reference mechanisms. Moon (2014a) indicates that such
threshold values or ranges can be established in two ways. First,
historical data can be explored to determine the appropriate threshold or
tolerance limits for a given KRI. For example, if prior data reveals that
late product deliveries of 1 percent or less is normal behavior from a
customer service perspective, then 1 percent might be used as the
benchmark in this case. Second, there are standardized KRIs and
thresholds established for certain industries. For instance, risk
consultancy rms have emerged in the marketplace that, among other
things, specialize in the development and provisioning of KRI measures
and associated benchmark information to their client bases.
In summary, it is critical that a comprehensive and leading set of KRIs
and corresponding thresholds or tolerance limits be developed and
maintained so the CRMA program reliably informs auditors and
management about the changing risk landscape. In this way, audit plans,
audit procedures and tests, and organizational risk management policies
and activities can all be addressed and revised in a proactive manner.
Auditor Response to Changing Risk Levels
The CEB Risk Management Council (2013) recommends the design and
implementation of a KRI dashboard system for the reporting of risk
information. A generic and simplied example of this is presented in
gure 6-3.
Figure 6-3: KRI Reporting Dashboard (taken from CEB 2013)
136
ESSAY 6: MANAGING RISK AND THE AUDIT PROCESS
In practice, the dashboard array might also include a variety of
visualizations, such as charts, graphs, and tables. Irrespective of content
and sophistication level, it is most important that both management and
auditors are provided with relevant, comprehensive, and up-to-date risk
information at a glance that is easy to interpret and able to be employed
in productively responding to changing risk patterns.
In the audit context, risk information would provide input for the
existing audit plan and potentially lead to a reformulation of audit
procedures and tests to be conducted. Vasarhelyi, Teeter, and Krahel
(2010) indicate that this dynamic approach will present problems for the
traditional auditor who tends to be resistant to change and demonstrate
rigidity relative to application of auditing standards. To be productive in
this new setting, auditors will need to be open to change, inclined to
incorporate new methods and technologies into the audit process as
warranted, and maintain a principles-oriented stance concerning
application of auditing standards.
Moon (2014a) proposes that the CRMA process be substantially
automated within the continuous auditing (CA) environment. In this
way, as KRI measures change and new signicant business risks are
identied, they are mapped by the CA system to corresponding audit
items and accounts vulnerable to material misstatement. Based upon the
outcomes, appropriate audit procedures are then executed to mitigate
audit risk. This would result in a real-time CA program equipped to
regularly update its settings and routines in accordance with information
generated by the CRMA module.
Management Response to Changing
Risk Levels
In CRMA, changes to KRI levels and signicant business risks also serve
as inputs to the organization’s risk management procedures and
processes. As such, the resulting information is instrumental in updating
the risk management plan. Incidentally, Moon (2014a) proposes that this
process should also become fully automated in a manner analogous to
the audit setting previously mentioned. Specically, as signicant
business risks are detected by the CRMA module, these items are linked
to the associated risk management protocols, and the set of risk
management procedures and processes are revised and implemented.
At this point, the primary features and mechanisms of CRMA have been
highlighted. As a complement to this discussion, it is useful to now
consider a simplied example of how such a system might function in
practice.
137
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
HYPOTHETICAL ILLUSTRATION OF
CRMA IN USE
Moxy Motors, Inc., an automobile manufacturer, maintains a robust
CRMA system. One noteworthy aspect entails continuous monitoring of
the regulatory environment relative to the risk that auto emissions
standards will become excessively stringent. In addition, given the
growing momentum concerning sustainability, social responsibility, and
"green" initiatives in general, mechanisms have been implemented to
monitor these areas as well. Consequently, there is a combined set of risk
indicators to report on the phenomena as they pertain to the automotive
industry. Although the overall reporting measure was in a normal state at
11:30 a.m., it suddenly spiked at noon so as to translate into a signicant
business risk. Fortunately, the operational managers and audit teams
were alerted to this problem immediately by the CRMA dashboard
reporting module, and promptly initiated investigations.
Management used the information primarily to revisit organizational risk
management protocols. A strategic response for addressing this severe
regulatory risk involved allocating additional research and development
(R&D) efforts toward the design and production of more
environmentally friendly vehicles such as hybrid fuel, electric, and
hydrogen-based cars and trucks. Furthermore, R&D emphasis regarding
traditional vehicle design and development was to be diminished by 25
percent. While these rearrangements would substantially increase costs
in the short-run, the estimated long-run benets provided more than
adequate justication for change.
Meanwhile, the auditors were simultaneously processing the current
CRMA information to rene the audit plan and adjust audit actions
accordingly. The growing concerns suggested that added audit
procedures and testing routines to mitigate audit risk should be enacted
and more explicitly emphasize accounts, balances, and transactions
potentially affected by R&D activities, discretionary accruals, and
revenue-related items. Because Moxy had relied predominantly on
production of fossil fuel burning cars and trucks, the burden placed on
the organization via restructured R&D investments could enhance the
probability of questionable accounting practices appearing in that area.
Furthermore, because of rising operating costs and resulting issues
relative to management’s concerns about meeting various short-term
earnings targets, there could be immediate pressure to manage earnings.
By proactively responding to the identied business risks, managers
productively implemented an action plan for addressing the emerging
problems. In addition, auditors were able to adjust the audit plan in
138
ESSAY 6: MANAGING RISK AND THE AUDIT PROCESS
real-time, thus improving condence that a high-quality audit could
ultimately be performed.
Systematic Implementation of Risk
Management and Assessment in a Process
Moon (2014b) outlines a formalized approach for the development and
implementation of a continuous risk monitoring and assessment
program. The corresponding framework is shown in gure 6-4, and
provides guidance in establishment of CRMA for a given process.
Figure 6-4: Framework for CRMA implementation (Moon, 2014b)
The diagram ow is essentially from left to right. However, pertinent
activities in the areas of communication, consulting, monitoring, and
reviewing will be ongoing during all phases of both system development
and usage (VMIA 2014). This facilitates continuous acquisition of
information and knowledge benecial for responding to identied issues
and rening the CRMA module as warranted.
When initially seeking to monitor and assess risk in a particular process,
a fundamental preliminary task is to identify relevant parties. More
specically, all internal and external stakeholders (that is, process
owners, subject matter experts, and auditors) should be identied by the
project team and included in the initiative so as to ensure program
functionality and success. Furthermore, the inclusion and involvement of
these stakeholders should be ongoing throughout development and
implementation phases of the CRMA initiative (VMIA 2014).
According to gure 6-4, business risk identication is the rst formal
step, and involves a comprehensive evaluation of the internal and
139
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
external environments. Initially, gaining a sufcient understanding of the
business is required, and entails considering organizational objectives
within the context of each target process. For example, if a
risk-monitoring and assessment routine is being established for the
manufacturing process, it would be necessary to envision how it
contributes to organizational objectives and the ways in which risks
might impact their achievement. Such an assessment will yield several
insights, including but not limited to process importance, extent of and
tolerance for associated risks, project scope, and resources likely required
for implementation of the risk monitoring initiative.
Following this, all relevant risks must be identied. According to the
VMIA (2014), this involves the creation of an exhaustive listing of items
that could affect the fulllment of organizational objectives. Referring to
historical documentation and consulting with pertinent stakeholder
groups will be a useful starting point. In addition, brainstorming,
interviews, surveys, and focus groups could produce supplemental input
for risk identication purposes. In documenting risks, it is also important
to specify how and why each risk might occur, take note of any existing
controls that mitigate risk realization, and consider associated
consequences of risk materialization. The emphasis in this stage should
be not only on documenting all possible risks, but also understanding the
root causes and outcomes of each risk event. For example, consider the
risk of information technology (IT) failure. One cause might be network
intrusion by an unauthorized external party. Among other things, one
consequence might be loss of sensitive customer data leading to customer
dissatisfaction and reputational damage. Certainly, management would
want to proactively anticipate and address such problems. From an
auditing perspective, risk of IT failure would be perceived as increasing
the probability of subsequent fraud. Therefore, audit efforts would be
appropriately modied in response to emergence of this risk event.
Once a complete listing of business risks is generated, emphasis is placed
upon generating KRIs to effectively monitor the risk environment.
Obtaining a rm understanding of risks and associated causes and
consequences in the previous stage should greatly assist with discovery
of a comprehensive and leading set of KRIs for subsequent monitoring
and assessment routines. Returning to the previous IT example, the
number of phishing attempts might be established as one KRI for risk of
IT failure. In addition, other metrics might be identied such that a set of
KRIs is meaningfully combined to collectively measure IT failure risk.
It is also imperative that, for each KRI, a suitable threshold or tolerance
range is determined. Otherwise, the implemented system will fail to
provide timely, relevant, and reliable warning signals concerning
emergence of business risks. To assist with this activity, prior experience
and documented industry KRI information can be relied upon. For
140
ESSAY 6: MANAGING RISK AND THE AUDIT PROCESS
instance, if historical ndings suggest that a product specications
conformity rate of 99.999 percent is minimally acceptable from both
product quality and customer satisfaction standpoints, then this might be
perceived as a legitimate threshold value. Performing this type of
analysis for each risk indicator will help ensure that functional threshold
values are structured so that management, internal auditors, and external
auditors are all able to proactively monitor and respond to the changing
risk environment.
Once KRIs and associated thresholds have been created and compiled by
the project team and relevant stakeholders, auditors begin linking each
KRI with the related subset of accounts and assertions to be tested (Moon
2014b). Objectives in this phase pertain mainly to understanding KRI
impacts on nancial statement information and, consequently, how audit
activities should be modied in response to changing KRI levels when
signicant business risks are detected. As an example, one important
audit assertion relates to valuation of assets. In an embedded component
of testing this assertion, the auditor will seek to determine the
reasonableness of any pertinent estimates such as depreciation,
amortization, and allowances. For instance, imagine that a KRI and
related set of tolerance limits based upon historical experience are
implemented to monitor depreciation levels on xed assets. When the
KRI moves near or outside the acceptable boundary, auditors are alerted
about an emerging risk relative to the valuation assertion for xed assets.
The accounts primarily affected in this case are non-current assets of a
depreciable nature. In addition, an income statement account (that is,
depreciation expense) is also affected, thus impacting net income in a
potentially questionable manner. In response to increasing risk in this
area, auditors would adjust the audit plan and testing activities so as to
more heavily emphasize examination of depreciable assets as well as the
reasonableness of depreciation amounts currently being recognized. If
problems of a material nature are unearthed, then timely remedies can be
proposed and implemented. In so doing, the auditor is positioned to
productively modify the audit plan and related testing routines
dynamically as changes in the risk landscape are reported by the CRMA
system.
The nal step in gure 6-4 encapsulates the implementation phase for
CRMA. In the auditing domain, the entire set of KRIs are continuously
measured and reported by the CRMA system. When emerging risks are
detected, auditors use this information to adjust the audit plan so that
appropriate emphasis is placed on examining and testing the assertions,
accounts, and controls that are most likely to be impacted.
To ensure ongoing system relevance and reliability, KRIs are regularly
maintained so that new measures are created, existing metrics are
modied, and obsolete KRIs are eliminated as the risk prole dictates.
141
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Furthermore, associated threshold values and tolerance limits are
addressed in a comparable manner. This requires continuous
communication and vigilance by process owners, subject matter experts,
management, and auditors, and is key to the long-run success of risk
monitoring and assessment. Given that the risk landscape is dynamic, an
effective monitoring system must be agile, and the CRMA approach can
provide for this capability.
CONCLUSION
We live in an interconnected world where the risk landscape and
business circumstances can and do change abruptly and unexpectedly.
The traditional risk identication setting, wherein risk assessment is done
as a discrete task and the risk landscape is not comprehensively
monitored in an ongoing manner, is increasingly untenable in our
evolving real-time global economy. Instead, a more uid approach is
necessary to monitor risks and facilitate modication of audit procedures
and testing routines, thus resulting in higher quality audits with
assurances that optimize the value added to corresponding information.
Although CRMA is admittedly in a formative stage, it offers genuine
promise in helping to achieving this vision.
REFERENCES
CEB Risk Management Leadership Council. "Develop SMART KRIs: Unlock the
Potential of Predictive Analytics." The Corporate Executive Board Company.
(2013).
Financial Times. Black Swan. (2014). Retrieved from http://lexicon.ft.com/
Term?term=black-swan.
Hill, C. W. L., and G. R. Jones. Strategic Management: An Integrated Approach (8th
ed.). Boston MA: Houghton Mifin Company, 2008.
Institute of Operational Risk. "Operational Risk Sound Practice Guidance: Key
Risk Indicators." (2010). Retrieved September 2014, from https://subscriber
.riskbusiness.com/ComponentFiles/Website/InterestingReading_Filename_
95.pdf.
Institute of Operational Risk. "Key Risk Indicators." (2014). Retrieved September
2014, from https://www.ior-institute.org/sound-practice-guidance/key-risk-
indicators.
Kuenkaikaew, S., and M. A. Vasarhelyi. "The Predictive Audit Framework." The
International Journal of Digital Accounting Research 13 (2013): 37–71.
Kuenkaikaew, S. Predictive Audit Analytics: Evolving to a New Era. Ph.D.
Dissertation, Rutgers Business School, Rutgers University. (2013).
Moon, D. "Continuous Risk Monitoring and Assessment (CRMA): New
Component of Continuous Auditing Systems." PhD dissertation (working
paper). Rutgers Business School, Rutgers University. (2014a)
142
ESSAY 6: MANAGING RISK AND THE AUDIT PROCESS
Moon, D. "CRMA Research Framework." Presentation. (2014b)
NACD. "Risk Governance: Balancing Risk and Reward." National Association of
Corporate Directors. (2009). Retrieved September 2014.
RepRisk. RepRisk website. (2014). Retrieved September 2014, from
www.reprisk.com.
Scandizzo, S. "Risk Mapping and Key Risk Indicators in Operational Risk
Management." Economic Notes 34(2) (2005): 231–256.
Vasarhelyi, M. A. "Continuous Assurance Presentation, Is the Thief Already Out
of the Barn?: Continuous Monitoring, Continuous Audit, and Forensics: What
Management Needs to Know." Rutgers Business School, Rutgers University.
(2011)
Vasarhelyi, M. A., M. Alles., and K. T. Williams. Continuous Assurance for the Now
Economy. Sydney, New South Wales, Australia: The Institute of Chartered
Accountants in Australia. (2010)
Vasarhelyi, M. A., R. A. Teeter, and J. P. Krahel. "Audit Education and the
Real-Time Economy." Issues in Accounting Education 25(3), (2010): 405–423.
VMIA. "Risk Management: Developing & Implementing a Risk Management
Framework." Victorian Managed Insurance Authority. (April, 2014).
Whittington, O. R., and K. Pany. Principles of Auditing & Other Assurance Services.
New York, NY: McGraw-Hill/Irwin, 2008.
143
PART II
Case Studies
145
CASE STUDY A
Developing
Continuous Assurance
at Siemens
Ann F. Medinets, MBA, PhD
Jason A. Gross, CPA, CIA, CFE, CISA, ACDA
Gerard (Rod) Brennan, CFE, PhD
Siemens Financial Services (SFS)
1
helps organizations in the energy,
construction, manufacturing, and healthcare industries to nance their
equipment and software, generate working capital, and manage their
portfolios and projects. SFS also provides nancing so that
municipalities may achieve their transportation and infrastructure goals.
In order to help clients reach their nancial targets, data integrity is
critical at SFS. Both internal managers and external customers need
assurance that the data used to make and evaluate strategic nancial
decisions is correct.
1
According to a recent press release,
Siemens Financial Services, Inc. (SFS) is the U.S. arm of the global Financial Services unit of
Siemens, which is an international provider of business-to-business nancial solutions. SFS
helps facilitate investments, providing commercial nance, project and structured nance
with specic asset expertise in the energy, healthcare, industry, and infrastructure and cities
markets. Employing more than 2,900 employees worldwide, SFS supports Siemens as well
as other companies with their capital needs and acts as an expert manager of nancial risks
within the Siemens Company. By leveraging our nancing expertise and our industrial know-
how we create value for our customers and help them strengthen their competitiveness. Be-
yond that, nancing is key in creating trust for technological solutions—and acts as a key en-
abler when it comes to the market launch. As of September 30, 2014, the total assets amounted
to €21.97 billion. For more information, visit: www.usa.siemens.com/nance.
147
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
TRADITIONAL INTERNAL AUDIT
In the past, SFS followed a traditional internal audit regime. Generally,
internal auditors act as an interface between the Board of Directors’
Audit Committee and the external auditors, so their responsibilities
relate primarily to the areas of internal control, risk management, and
governance.
Internal control: All organizations are supposed to maintain a system of
internal controls to accomplish four key goals: (1) to monitor the
efciency and effectiveness of the organization’s operations; (2) to
safeguard the organization’s assets; (3) to prevent errors that might
impair the reliability of the organization’s nancial and managerial
reporting, or to detect and correct such errors once they occur; and (4) to
ensure the organization’s compliance with relevant laws and regulations.
Although organizations may choose different methodologies for specic
tasks, the internal auditors or the internal control functions typically
monitor, evaluate, and make recommendations to management in order
to improve the network of internal control policies and procedures
designed to accomplish these goals.
Risk management: In the area of risk management, internal auditors
work with managers to assess and mitigate risks that might impair the
organization’s business strategy under the COSO (Committee of
Sponsoring Organizations of the Treadway Commission) enterprise risk
management (ERM) framework. In addition, as part of the organization’s
compliance with Section 404 of the Sarbanes-Oxley Act (SOX), internal
auditors report to management on any weaknesses in the organization’s
internal controls that might create a potential risk of a material
misstatement in the organization’s nancial reporting.
Governance: The COSO ERM framework broadly denes governance to
include all activities related to directing the organization’s operations in
order to achieve its operational goals and protect stakeholders’ interests.
The governance aspect of the internal auditor’s job is to work with the
audit committee on control issues and to facilitate the ow of information
between the external auditor and the audit committee.
CONTINUOUS CONTROLS MONITORING
The skills and techniques used in internal audit can also be applied to
continuous controls monitoring (CCM) programs. The vice president of
internal audit started at SFS in 2002, but starting in 2009, the company
leveraged his years of knowledge of its internal business processes and
control framework to develop the company’s CCM program. In his new
148
CASE STUDY A: DEVELOPING CONTINUOUS ASSURANCE AT SIEMENS
role as vice president of controls management at SFS, he observed that
many of the company’s internal audit programs, such as audit checklists,
are mechanical and repetitive in nature, so he wanted to automate these
processes by moving to a CCM program as an independent control
assurance function that would be separate from internal audit.
Recognizing that organizations are often resistant to change, he decided
to start with small, achievable targets to demonstrate that the program
has merit, and then expand beyond nancial reporting into operational
components once the business unit and support unit managers at SFS
had become accustomed to the continuous monitoring methodology.
In the rst phase of the project, the new vice president of controls
management focused on nancial reporting rules compliance and data
integrity. Personnel at SFS manage data and make decisions within a
framework of company policies expressed as a set of rules. Because SFS
relies on communicating rules and assessing compliance with those rules
in providing services to its clients, the rst part of the project was to
develop a system that could notify key people within the company about
"exceptions" and "alerts." An exception demands immediate research and
resolution because it means that a rule was not followed. Depending on
the explanation that is given for the exception, it might also require a
correcting entry in the SFS accounting records. By contrast, an alert ags
any transaction that might be of interest to the owners of that
information, such as a change in a transaction, so that SFS can take
immediate action to verify the information and be proactive in
monitoring the business.
Implementing a CCM program would enhance business processes at SFS
by immediately identifying any exceptions to the company’s system of
rules and policies by internal decision-makers, notifying them of these
exceptions, and demanding resolution. The CCM system would also
improve the company’s nancial assurance processes, such as SOX
requirements, by monitoring the entire data population, instead of
relying on the types of sample testing used in traditional SOX or internal
audit methodologies. According to the vice president of controls
management, "Exceptions in the data pool are like sh in the lake. Just
because they are there doesn’t mean you will catch any." Continuously
assessing 100 percent of data attributes (validity, authorization,
completeness, valuation, time period, and disclosure) for exceptions
gives far greater assurance that the data represents the company’s
underlying economic position than periodic sampling ever could. In
addition, the immediacy of the CCM’s feedback raises decision-makers’
cultural and behavioral awareness of rules at SFS, which further
enhances its value as a tool and a methodology.
149
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Designing and instituting a CCM program requires nding a balance
between complexity and exibility. It must be simple enough for all
personnel to use it correctly, but capable of being tailored to the needs of
customized data sets, disparate system platforms, and unique business
rules. For the CCM program at SFS, the vice president of controls
management chose the ACL Analytics Exchange (AX) platform,
including AX Exception and ACL Analytics. At the most basic level, this
system recursively tests 100 percent of transactional data in the
company’s sub-ledger system for anomalies and control failures, and
noties the owners of the data about any alerts or exceptions within their
area of responsibility. As a result, process owners would get a higher level
of assurance on data integrity. In addition, process owners and members
of the controls management team can be freed from repetitious manual
tasks and time constraints. Therefore, they can refocus their skills and
energy on specically agged areas, and skip information in parts of the
business processes that do not need immediate attention. As an added
benet, the number of exceptions detected was expected to decrease
over time. Many exceptions come simply from habit, so better
communication of the rules and mechanisms should reduce or eliminate
repetitive problems, resulting in overall consistency and process
improvement.
The data analytics manager at SFS provided a sample of three of the
company’s departments, covering approximately 60 of the 150
exception-type testing algorithms (commonly referred to as "analytics")
in the scope of the CCM program. He noted the substantial
improvements in the average percentage exception rate from the
program’s inception to the present. These analytics cover various
attributes including data input checks, validity checks, and compliance
with regulations and internal policies. As shown in gure A-1, the
consistent reduction in the average exception rates demonstrates an
adherence to controls driven by the continuous monitoring program. For
instance, Department A showed a reduction in percentage exception rate
from 12–14 percent in 2011 and 2012 to 2–4 percent in 2013 and 2014.
Even more strikingly, Department B improved from an exception rate of
21–25 percent in 2011 and 2012 to 2–3 percent in 2013 and 2014, and
Department C improved from an exception rate of 4–6 percent in 2011
and 2012 to less than 1 percent in 2013 and 2014.
150
CASE STUDY A: DEVELOPING CONTINUOUS ASSURANCE AT SIEMENS
Figure A-1: Change in Departmental Exception Rates at SFS (2011–2014)
SFS currently runs approximately 250 exception-type and alert-type
analytics on a daily basis. The results of this sample indicate that the
exception rates have signicantly improved and remained consistent
since the implementation of the CCM program into production in
February 2010. The vice president of controls management has observed
that departments are starting to request that new analytics be added to
the CCM program as the business unit and support unit managers at SFS
recognize the system’s value.
In addition to the analytics, the vice president of controls management
made some design choices to customize the ACL AX system to specic
needs and workows at SFS. He decided to have the system report an
aging of exceptions. This forces the owner of each exception to deal with
the problem in a timely fashion. He also chose to implement closed-loop
escalated alerting. One of the most powerful features of the system, these
alerts operate like trafc cameras at red lights, sending a "ticket" up the
company’s hierarchy if the owner of the exception fails to address the
issue. Finally, he determined that there should be no unilateral closing of
unxed problems. Each exception must either be xed or explained by
the owner and then reviewed and veried by someone else. In terms of
data integrity, this means that if the owner closes an exception on the
web-based system, it must also be validated by the CCM system. Thus,
the exception must be resolved fully in the sub-ledgers because the
owner of the data cannot simply make the problem disappear. Otherwise,
the exception will be republished the next day, and the CCM program
151
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
will also notify higher-ups. The goal of this procedure is to ensure that all
exceptions are corrected, but this design choice also has managerial
implications. It helps supervisors to see the types and frequencies of
exceptions that are occurring so that they can consider whether their
subordinates are taking too many liberties with the rules, or whether the
rules themselves might need to be changed. Alternatively, particular
patterns of exceptions might indicate that specic processes may need to
be improved.
In 2013, the vice president of controls management instituted a valuable
innovation to the CCM program. Because the CCM program already
automatically identies new exceptions daily, he realized that the same
approach can be applied to notify the owners of the analytics, on a daily
basis, when their previously agged analytic exceptions have been fully
corrected and resolved in the sub-ledger. Therefore, noteworthy
efciencies in the CCM process became achievable. This drove the
creation of a CCM functionality that the vice president of controls
management refers to as "Auto Close." The Auto Close feature eliminates
the requirement for the owner of an analytic to update the status of a
corrected and closed exception manually on the website. The benets of
Auto Close include avoiding the manual step of closing the items on the
AX Exception website, typing comments, and/or uploading supporting
documentation. In addition, this new process ensures a real-time status of
corrected items on the website for improved transparency of unresolved
items because, at times, the step of updating the AX Exception website
manually may not be timely or fully maintained. The Auto Close feature
also ensures strong change management through logging of the data
corrections on a before-and-after view and archives this information as
an attachment in the history audit trail in the AX Exception website. Auto
Close conrmations are automatically emailed to the analytic owners as
they are identied by the CCM Program.
The vice president of controls management indicates that, although most
people think of a CCM program as a detective control, the benets of the
CCM program reach further if planned and incorporated carefully. The
program can also be a preventative control because it monitors
transactions as they occur during the month, and identies exceptions
daily (or in real-time) and corrects them immediately (or at least before
the end of the month). Therefore, the exceptions are corrected in the
sub-ledger prior to the month-end closing process. As shown in the
following diagram, this eliminates the need for the accounting function
to book correcting journal entries manually and results in a clean general
ledger without manual adjustments. The vice president of controls
management says, "It’s a win-win for the business operations and the
accounting function. We can target common sources of correcting journal
entries as analytic targets in the CCM program."
152
CASE STUDY A: DEVELOPING CONTINUOUS ASSURANCE AT SIEMENS
Figure A-2: CCM as a Preventive Control to Improve the Closing
Process
Of course, the way that the CCM functions within an organization’s
internal control system depends on the specic structure of the
organization’s bookkeeping system. For example, if an organization posts
to the general ledger monthly, then the CCM operates as a preventive
control because it catches errors before they are posted. Alternatively, if
transactions are posted daily, then the CCM serves as a detective and
corrective control because it xes errors after they are posted.
To a large extent, internal control specialists are the architects of an
organization’s analytic tools because they have a unique awareness of the
risks and controls, as well as the kinds of information and the
information formats that will be needed by managers for
decision-making and by internal and external auditors for risk
assessment and assurance purposes. For example, the AX system has a
naturally auditable log function that constantly tracks actions performed
in the CCM program to provide a solid trail in one direction to the way
that exceptions were derived, and in the other direction to subordinate
clean results that may be relied upon by both internal and external
auditors. This creates an audit trail that facilitates both internal and
external audit activities. Thus, the AX log not only nds exceptions and
monitors their resolution, but also reports on exceptions and keeps a log
of clean results that serve as evidence on the correctness of the residual
data. As an added benet, this is a tool that the external auditors easily
understand and trust because they already use this software application
in their work. AX has also streamlined the processes necessary to comply
with requirements for internal controls over nancial reporting under
SOX and ICFR (internal control over nancial reporting). SFS no longer
needs to sample the functionality of selected internal controls that have
already been migrated to CCM for Section 404 reporting because the
system has already tested 100 percent of the population, giving the
external auditors a high level of reliance on the data as they plan the
timing, nature, and scope of the audit. It also eases the client’s burden to
provide sample-based data to the auditors and streamlines or eliminates
the need for management assurance testing.
The vice president of controls management sees the CCM system
primarily as a tool of management. For each managerial decision, the
153
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
system enables managers to prepare the accounting for that decision,
tracks ownership of the data related to that decision, and provides a
check on whether the company’s rules were followed for each attribute of
that decision. In addition, the system helps owners to do spot checks and
ag potential issues.
At SFS, the controls management department owns and controls the
ability to modify or add new analytics to the ACL CCM program, and the
company’s managers own the results and resolutions to exceptions, but
the rules are the core of the system. They need to be expressed in a way
that is system-compatible, and must be monitored to ensure that they are
operating as intended. The ACL Analytics Exchange is a highly exible
system that can include components such as automated fraud detection
and forward-looking data analytics. Its only limitations are an
organization’s available resources.
Another potential area for expansion is strategic data analysis. Specic
identication is the most granular level of data, but it is not always the
most useful for planning the future. The North America risk and internal
control ofcer at Siemens points out that the ACL Analytics Exchange is
compatible with systems like SAP HANA, a powerful platform that
offers the capability of running analytics for the CCM system (or any
process) much faster because it optimizes data management so that the
data is, in effect, stored in random access memory (RAM). This allows
"in-memory" analytics to be run instantly with real time results. Users no
longer need to search remote databases to nd and transport data,
improving the latency and response time of analytics. This is
breakthrough technology in the world of accounting, auditing, and
monitoring because it allows for the fast and efcient use of analytics
across multiple databases on structured and unstructured "big data" with
a signicant reduction in complexity and with instant results.
CCM is a focus for Siemens across many corporate functions, business
units, products, and processes. Analytics, as a key support for CCM, is an
important enabler of the company’s future vision for control, but it also
helps to drive the use of more automated common process in all business
areas. This technology can be used to help ensure that people are
following dened processes and strategies, to prevent or detect fraud, to
improve project management, to monitor key risk indicators and
opportunities, and as an early warning system to prevent problems
before they happen. According to the North America risk and internal
control ofcer, "We have made good progress in some areas and are just
conceptualizing opportunities in others. For the future, we need a clear,
integrated enterprise strategy around continuous monitoring and
common, easy-to-use tools across all business units and regions to best
leverage the power of this technology to improve business processes and
reduce costs. Continuous monitoring and the use of analytics is a
154
CASE STUDY A: DEVELOPING CONTINUOUS ASSURANCE AT SIEMENS
‘disruptive change’ that can and will have a signicant impact on
improving products and improving business and operational processes
in the future."
The vice president of controls management believes that the system
developed at SFS can be leveraged into a scalable, sustainable, replicable
application for other parts of the Siemens organization. Ultimately, his
goal is to move the company from computerized, continuous, internal
monitoring to a fully integrated, real-time, continuous assurance system
for internal control.
155
CASE STUDY B
Implementing
Continuous Auditing
and Continuous
Monitoring in
Metcash—Change,
Capabilities, and
Culture
Glen Laslett, CA, CIA
Catherine Hardy, PhD
INTRODUCTION
Developing continuous auditing (CA) and continuous monitoring (CM)
capabilities are consistently recognized in industry surveys as a top
priority for not only improving internal audit effectiveness and efciency
but also for adding value in increasingly complex and changing business
environments (KPMG 2012a; Protiviti 2014; PwC 2014). While the
benets and business imperative for CA/CM are widely accepted,
uncertainty remains as to how this can be effectively accomplished
(Hardy 2014). Building a sustainable CA/CM system requires far more
than a business-as-usual type approach and "adding some tactical and
157
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
technical data analytic capabilities" (KPMG 2013, 1). A number of papers
have referred to the challenges in being able to effectively leverage data
analytics and CA/CM techniques (PwC 2014) and then successfully
integrating them into the audit process (KPMG 2013). We argue that the
principal challenge is more fundamental. A major issue facing internal
audit functions is developing an architecture that facilitates the regular
capture and management of data from various systems, as well as the
development and reliable delivery of continuous monitoring routines
(CMRs) that efciently test controls and data as well as mechanisms that
allow the tracking and analysis of exceptions.
We draw on the Metcash case, an Australian-based company that has
reached an advanced maturity stage over a decade of development in
using CA/CM (Hardy and Laslett, forthcoming). Key issues and
challenges faced by the internal audit group when implementing and
using CA/CM are described, as well as the lessons learned. More
importantly, the case also reveals how CA/CM is being used to transform
internal auditing and future directions. Our discussion commences with
the value proposition for adopting CA/CM in Metcash. A description of
the architecture follows. The types of CA/CM applications used are
outlined, giving detailed attention to one specic CMR due to its relative
uniqueness as well as its coverage of the core elements of value-added
CMR development: multiple data sources, complex algorithms, and
accountable exception delivery. The implication for implementing
CA/CM and internal audit more broadly concludes the case discussion.
VALUE PROPOSITION:IDENTIFYING THE NEED
AND
ADDRESSING THE BUSINESS CHALLENGE
The value of CA/CM for Metcash needs to be placed in the context of the
company’s history and overall strategy. Metcash operates in four market
areas: food and grocery, liquor, hardware, and automotive. The vision of
the company is to grow its markets and deliver value to its stakeholders,
provide excellence in distribution and merchandise, be retailer and
consumer champions, and be a place of choice to work. The company has
had a long history of growth through acquisitions over a 40-year period.
This was a period of signicant change, resulting in organizational
restructures, a new senior management group and diverse systems.
Investments in enterprise platforms relating to warehousing and
nancial systems have occurred over the past decade.
There were clear benets to be accrued by implementing a CA/CM
system. Metcash processes high volumes of transactions. For example,
the accounts payable department processes approximately 760,000
158
CASE STUDY B: IMPLEMENTING CONTINUOUS AUDITING AND CONTINUOUSMONITORING
invoices annually, with a further 1,500,000 processed for direct deliveries
to customer stores. Accounts Receivable processes nearly 700,000
transactions each month (Hardy and Laslett, forthcoming). An
automated environment for testing 100 percent of controls and
transactions on a near real-time basis was always going to bring value to
the business if executed appropriately. However, the benets to be gained
were far more than simply automating manual tasks. Closer alignment
with management needs, reduction of audit costs, improved focus in
audit planning and development of a superior data analytics capability
provided added value in terms of cost, competencies, and relationships.
There was not an explicit focus on these value adding activities in the
early stages of implementing CA/CM. Rather, they emerged over time,
building on incremental successes of rolling out applications of CA/CM
commencing with routine tasks such as duplicate invoices and
progressing to more sophisticated applications that are discussed later.
The benets delivered so far have exceeded the cost of achieving them.
However, the implementation of CA/AM was not without challenges.
Long lead times, complex technical environments and a commitment of
resources meant that the range of benets were not easily recognizable in
the early stages. However, savings from automating routine audit
procedures, (for example, potentially duplicated invoice processing),
provided an "early win" for demonstrating value to management. In the
earlier stages of developing and operating the automated testing there
was political resistance from some business owners about the value of
adopting the CA/CM system. Active leadership was required from the
internal audit group to manage not only the technical issues, but also the
change process in terms of managing varying perceptions and
expectations about how exceptions would be identied and managed;
impact on work routines; and the need for collaborative efforts in
designing scripts and using technologies that would best assist in
providing greater insights into business activities. Extensive consultation
was needed to ensure user engagement and that the benets were
meaningful to the business owners. This transformative approach
assisted in focusing attention on business improvement outcomes, thus,
changing behaviours and demonstrating the value that internal audit can
bring to the business.
THE IMPORTANCE OF ARCHITECTURE
The CA/CM solution resembles data warehouse solutions in that data is
"extracted, transformed, and loaded" into the CA/CM applications.
159
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure B-1: The Architecture
(Source: Hardy and Laslett, forthcoming)
Commercially available software is used, incorporating ACL, CaseWare
Monitor™, ACL Direct Link™ for SAP, Windows Scheduler, and Excel
spreadsheets, each serving different purposes (Hardy and Laslett,
forthcoming). A CA/CM system based on spreadsheets is not
sustainable. A robust architecture is required that supports the
identication of exceptions, and provides assurances that the exceptions
have been actioned by relevant parties and that the exception data is
retained and secured appropriately. The need for a "robust" database to
manage and store large and growing amounts of data is not new (Alles,
et al. 2008, 150). However, it was a challenge for the internal audit group
as they were responsible for the acquisition, maintenance, and security of
the CA/CM infrastructure. The IT department viewed this as an
end-user responsibility.
The architecture consists of three component areas: (1) data extraction; (2)
data transformation; and (3) loading of exception data into user
accessible exception management software. This is broadly based on a
monitoring and control layer (MCL) type of approach (Vasarhelyi et al.
2004). As shown in gure B-1, ACL Direct Link™ is used to extract data
from the ERP (SAP) system. Non-SAP data requires other extraction
techniques. ACL™ is used to transform different native le formats and
execute the scripts. Experience has shown that vendor pre-written scripts
160
CASE STUDY B: IMPLEMENTING CONTINUOUS AUDITING AND CONTINUOUSMONITORING
cannot be universally applied because business units have unique
business rules. Exceptions are managed using the CaseWare Monitor™
software. The software supports notications of exceptions, the person
responsible, actions taken, the required time frame and when it has been
resolved. Movement and pending reports have been developed out of
CaseWare Monitor™ to better analyse the queues. By having greater
visibility over the resolution process, the internal audit group is able to
focus its attention on higher level oversight.
DEFINITIONS AND APPLICATIONS OF CA/CM
IN METCASH
There is a range of denitions used for CA/CM (Hardy 2014). While
some distinctions are based on roles, this was not a signicant issue in the
early stages of development in Metcash. The internal audit group led the
development of the CM system. Responsibility for the CM system was
handed over to the business owners after it was fully functioning. At this
stage, the internal audit department played more of an oversight role,
examining trends and reviewing whether business owners were using the
system as intended. This approach supports the view that neither CA or
CM needs "to be present for the other to be implemented" (KPMG 2012b,
3), but by combining them there is a more efcient use of technical and
human resources through coordinated efforts (KPMG 2012b, 3). However,
similar to Vasarhelyi et al. (2012, 275), ndings, the monitoring systems
shared between audit and management at the full continuous audit stage
were not being fully used by management. Internal audit was generating
exception reports and monitoring results that were being passed on to
management. There was a degree of uncertainty as to when and how
these exceptions were being acted on. Metcash developed a complete
exception management system to monitor and determine how exceptions
are being actioned by management using the CaseWare Monitor™
workow application. The architecture is discussed in the following.
Denitions of CM that emphasise how weaknesses in control
designs or operations can be identied may also cause some level
of confusion. It is challenging to monitor controls directly. For
example, a review of segregation of duties (SOD) violations may
reveal conicts, but it does not address mistakes or possible fraud
of authorised users. The continuous review of master data and
transaction exceptions allows inferences to be drawn regarding the
effectiveness of the controls and where gaps may exist. For
example, Metcash policy requires staff to declare related party
interests. In order to conrm that this policy is effective, a CMR
compares changes to the vendor master le to the employee
master le on a daily basis. Matches between the master les on
161
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
sensitive data elements (address, telephone numbers, and the like)
trigger exceptions for the review of the Group Security function.
Multiple dimensions need to be considered when integrating
CA/CM and analytics, incorporating transactions, controls and a
"macro-analytic" aspect such as differences in metrics and patterns
within a business unit or across an organization (KPMG 2012b, 6).
An Example Application: The Leave
Continuous Monitoring Routine (CMR)
Metcash currently has more than 100 applications in a range of areas (see
Hardy and Laslett, forthcoming for a list). One of these CMRs is
relatively unique. All business entities struggle with ensuring that all
leave (that is, compensated absences including sick, vacation, and other)
is properly accounted for. Failures to complete compensated absence
applications, particularly in large organizations, may lead to overstated
compensated absence liabilities and adverse effects on cash ow and
operations as staff leave the business or take compensated absences to
which they are unentitled.
A brief description of the leave CMR algorithm follows:
r
Log-in data is captured from all sources across the business.
r
Any staff member who has not logged in for two or more business
days is identied and the exception is cross-matched to the human
resources leave records.
r
If no leave record is found, the exception is agged, and the CMR
continues to check the leave records for the next 14 days.
r
If no leave record is processed during the 14-day window, the
exception is processed to the exception management workow
system.
r
An automated email (in the name of the responsible HR manager)
is sent to the employee’s manager alerting him or her to the gap in
the log-in records and asking him or her to review the exception
and advise the human resources team.
r
Once a response is received, the exception is closed in Caseware
Monitor
TM
including a reason and action code for further analysis
and reporting.
r
If the exception remains unclosed, the CMR continues to produce
follow-up emails asking the manager to review the exception and
respond accordingly.
r
For those exceptions closed as "leave owing" in Caseware
MonitorTM, the CMR subsequently checks to ensure that the leave
162
CASE STUDY B: IMPLEMENTING CONTINUOUS AUDITING AND CONTINUOUSMONITORING
has actually been processed as promised. Exceptions are passed to
Human Resources for further review.
r
The CMR also examines cases where exceptions have been closed
as "no leave owing," but leave is processed nonetheless (as a test of
integrity).
r
The CMR produces weekly activity and status summaries for
executive management.
The leave CMR is complex. We argue that it tests the boundaries of
CA/CM and thus provides an illustration of developments in an
environment with multiple data sources and a complex software and
data ecosystem. Specically, the CMR:
r
accesses several systems to assemble and cross match the data;
r
applies relatively complex algorithms to reduce the number of
false positives; and
r
uses multiple platforms to deliver and monitor the exceptions
including ensuring that the exceptions are being actioned as
advised.
The value of CA/CM is diluted unless there is a robust mechanism to
track and resolve exceptions. Further, the value of CA/CM is also
reduced if the algorithms do not effectively address the suppression of
false positives. Experience suggests that external auditors tend to nd
techniques of this sort to be uneconomical due to the need to incorporate
the business rules of differing clients in the algorithms. For example,
tools such as ACL can easily check for potential duplicate invoicing;
however, ACL will potentially produce large numbers of potential
exceptions unless scripts are developed to, for example, identify
matching reversals. The experience at Metcash has been that multiple
iterations of the algorithms are required to minimise false positives.
Moving Forward—Key Risk Indicators
Metcash has moved away from the AS/NZS ISO 31000 Risk Management
Standard risk proling approach, as published by the International
Standards Organization. A static 5 × 5 matrix that builds on likelihood
and consequences is not consistently of practical use to management.
Monitoring and assessing key risks through data driven risk indicators
provides a greater benet to management. The increasing availability of
data and sophisticated analytics has facilitated a more accurate
identication of problems in critical areas such as Food Safety and
Human Resources. The use of dashboards (see gure B-2 for example)
has enabled the risk indicator data to be effectively communicated to
management via various media including tablet devices.
163
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Figure B-2: Example of a KRI Dashboard: Dummy Data
CHALLENGES AND LESSONS LEARNED
Eight key challenges were encountered and lessons were learned:
r
Implementing a CA/CM system is not a trivial exercise. While
CA/CM has a high payback, there are long lead times and a
signicant investment of time and effort is required.
r
Data management is critical in ensuring that the right data is
accessible on a timely basis. However, data accessibility and
quality issues were major inhibitors in the early stages of
implementing CA/CM at Metcash.
r
CA/CM needs to be robust, sustainable, and deliverable. This
outcome involves the development of adequate physical
architecture, data coding standards, documentation, exception
management, and backup. As an end-user development, the
CA/CM system requires a range of controls around it to ensure
that it continues to operate properly and maintain its integrity.
More than 100 fully automated scripts run on a daily basis, some
of which are quite complex. It is vital that these applications are
properly managed.
164
CASE STUDY B: IMPLEMENTING CONTINUOUS AUDITING AND CONTINUOUSMONITORING
r
In-house development requires ongoing focus over at least two to
three years. Outsourcing may be a viable option to build a CA/CM
system in the short-term and particularly so with respect to
architecture development. A third-party developer may be able to
build CA/CM applications within a "green eld" site relatively
quickly and may be able to provide this service as a "cloud" based
service.
r
Implementation does not tend to progress in discrete linear steps
as normally represented in current guidance. Rather, the CA/CM
environment tends to unfold through an iterative and incremental
process with key learnings being acquired in layers.
r
"Meta" CM routines may be specied, but it is rare that a high
value-add can be derived from generic CM routines due to
different business rules and data formats. Caution should be taken
with "potted solutions" offered by third parties, as large volumes
of false positives can arise. The case of those opposing the
development of CA/CM for whatever purposes will be
strengthened by the delivery of unusable volumes of exceptions,
most of which prove to be false positives. Metcash’s experience is
that much of the coding surrounding the CMRs is dedicated to
removing false positives so as to ensure that the output is largely
composed of "high probability exceptions."
r
The downside of success with a CA/CM system is that it may
result in it becoming a de facto production system. CA/CM tends
to become part of the production landscape as users begin to rely
on the output. Internal audit begins to interact closely with the
business and responds to their demands for further development.
In that context, particularly from a CM perspective, the
environment must be adequately controlled. Examples include
controls to report failed CMRs, complete data capture as well as
regular back-ups, coding standards, and periodic manual review
of output to ensure that the output continues to be complete and
reliable. For example, Metcash has written several hundred
thousand lines of code in the context of its CA/CM system. The
loss of this code would be catastrophic. Rigorous backup
procedures are followed. The operation of a powerful set of CM
applications may be viewed by some as potentially eroding the
independence and objectivity of the internal auditors. It is our
view that "objectivity" is a state of mind and, provided that the
internal auditors do not subordinate their judgment to that of the
end users, any potential conicts can be effectively mitigated.
r
Exceptions must be delivered reliably and effectively followed up.
The value of CA/CM would be seriously diluted if there is a lack
of comfort that control breakdowns are remedied and
transactional exceptions are investigated and corrected as
165
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
necessary (for example, duplicate invoice processing). A failure to
properly manage exceptions will adversely affect stakeholder
engagement, the opportunity to change behaviors, and the
realisation of benets.
CONCLUSION
CA/CM has evolved into a valuable system in Metcash. Its success
depends on having the right talent, technologies, and culture. Data
management, technical expertise (for example, analytics, enterprise
systems, and audit knowledge and capabilities), and managing change
and culture were the main elements in building Metcash’s capability. The
CA/CM system grew organically for more than a decade. The head of the
internal audit group played an ongoing and signicant role in building
this capability. The immediate value of CA/CM may not be entirely
evident to the business owners in the early stages of implementation.
Further, CA/CM presents new ways of doing things, challenging existing
practices. An experienced and senior audit professional is required to
enlist senior management support in managing the transition. In
addition, internal audit needs to invest signicant effort in collaborating
with business owners to work towards a common goal and use insights
garnered along the way to guide future development. The Metcash
culture was open to new ideas and approaches, having experienced
signicant organizational change over a long period. These "norms" and
values assisted with internal audit’s efforts in implementing CA/CM.
The advances made through CA/CM are transforming the way internal
audit is conducted in Metcash as well as how it is perceived. By
successfully leveraging technology, internal audit is able to contribute far
beyond reviewing past activities, additionally providing insights for
business performance and value-added activities. In uncertain and
complex business environments, this presents an opportunity for internal
audit to become a key partner in managing risk and guiding success.
REFERENCES
Alles, M.G., A. Kogan, and M.A. Vasarhelyi. "Putting Continuous Auditing
Theory into Practice: Lessons from Two Pilot Implementations." Journal of
Information Systems 22(2) (2008): 195–214.
Hardy, C.A. "The Messy Matters of Continuous Assurance: Findings from
Exploratory Research in Australia." Journal of Information Systems (2014).
Hardy, C.A. and G. Laslett. "Continuous Auditing and Monitoring in Practice:
Lessons from Metcash’s Business Assurance Group." Journal of Information
Systems (forthcoming). Journal of Information Systems (JIS) early online
manuscripts DOI: 10.2308/isys-50969.
166
CASE STUDY B: IMPLEMENTING CONTINUOUS AUDITING AND CONTINUOUSMONITORING
KPMG. "Continuous Auditing and Continuous Monitoring: The Current Status
and the Road Ahead. (2012a). Retrieved from www.kpmg.com/us/en/
services/advisory/risk-and-compliance/internal-audit-risk-and-regulatory-
compliance/pages/continuous-auditing-continuous-monitoring.aspx.
——. "Leveraging Data Analytics and Continuous Auditing Processes for
Improved Audit Planning, Effectiveness, and Efciency." (2012b). Retrieved
from www.kpmg.com/US/en/IssuesAndInsights/ArticlesPublications/
Documents/data-analytics-continuous-auditing.pdf.
——. "Transforming Internal Audit: A Maturity Model from Data Analytics to
Continuous Assurance." (2013). Retrieved from www.kpmg.com/US/en/
services/Advisory/risk-and-compliance/internal-audit-risk-and-regulatory-
compliance/Documents/transforming-internal-audit.pdf.
Protiviti
®
. "Assessing the Top Priorities for Internal Audit Functions, 2014
Internal Audit Capabilities and Needs Survey." (2014). Retrieved from
www.protiviti.com/en-AU/Pages/IA-Capabilities-and-Needs-Survey.aspx.
Pricewaterhouse Coopers. 2014 State of the Internal Audit Profession Study,
Higher Performance by Design: A Blueprint for Change. (2014). Retrieved from
www.pwc.com/us/en/risk-assurance-services/publications/pwc-2014-state-
of-profession.jhtml.
Vasarhelyi, M.A., M.G. Alles, and A. Kogan. "Principles of Analytic Monitoring
for Continuous Assurance. Journal of Emerging Technologies in Accounting 1
(2004): 1–21.
Vasarhelyi, M.A., M. Alles, S. Kuenkaikaew, J. Littley. "The Acceptance and
Adoption of Continuous Auditing by Internal Auditors: A Micro Analysis."
International Journal of Accounting Information Systems. 13 (2012): 267–281.
167
CASE STUDY C
Increasing Audit
Efciency Through
Continuous Branch
KPI Monitoring
1
Carlos Elder de Aquino
Eduardo Miyaki, CIA, CCSA, CFSA
Nilton Sigolo
Miklos A. Vasarhelyi, PhD
Paul E. Byrnes, CMA
ABSTRACT
Over the last decade, a large South American bank (SAB) has monitored
over 1,400 retail branches and been involved in a remarkable
leading-edge transformation of the nature of its audit process.
Historically, an annual 160-hour audit was performed for each branch,
and the associated work was either conducted by the internal audit staff
or outsourced to one of the large audit rms. During the previous ve
years, this process was reengineered to perform daily continuous
monitoring in conjunction with an annual potential 40-hour surprise
audit of each branch. The banking system in this country is progressive in
that it maintains overnight clearing processes and superior information
technology (IT) and internal controls. This case study can provide U.S.
corporations with unique and valuable insight regarding how a nimbler
and more advanced set of audit processes can be implemented.
1
http://raw.rutgers.edu
169
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
INTRODUCTION
The audit profession has accelerated adoption of continuous auditing
and assurance mechanisms now that attention to Sarbanes Oxley and
other compliance activities have been embedded into existing processes.
Since 1999 (CICA/AICPA 1999), additional guidance has been issued by
the Institute of Internal Auditors (IIA 2005) and ISACA (ISACA 2012), but
substantial conceptual confusion nevertheless persists in this domain.
This case study uses an actual implementation example to illustrate and
clarify the conceptual issues as well as suggest some plausible solutions.
Country regulations and internal policies obligate the bank to perform a
yearly branch audit for each of its more than 1,400 locations. Each branch
audit historically entails about 160 hours of audit work, requiring the
outsourcing of many of these engagements to external auditors at
substantial expense and with nontrivial logistical challenges. A
productive solution to this problem involved overnight monitoring of
each branch using 18 indexes as well as the adoption of an annual
potential 40-hour surprise audit of each branch. Monitoring in this case
was performed by a "continuous audit" process in which variances from
standards were regularly measured and reviewed. When a variance was
deemed to be outside the range of acceptability, an email message
prompting review and explanation of the event was sent to regional
managers (to whom branch managers reported). This strategy has proven
to be cost effective and provides for process improvements and enhanced
allocation of scarce resources. The solution is also estimated to be
generating substantial savings for the bank.
THE PROCESS AT SAB
Continuous monitoring and assurance at SAB follows the generic outline
displayed in gure C-1. More specically, a nightly extraction routine is
executed and captures 18 overall process key performance indicators
(KPIs) and relevant transaction dimensions such as amount, timing,
nature, and branch. Accumulated values are then compared to historical
data, company standards, and master les containing client
characteristics, parameters, and other factors. In general, the standards
employed create an average of about 800 exceptions per week that are
subject to initial screening on an individual basis. Of these, about half are
ultimately sent for further action to a regional manager who supervises
individual branch managers. To allow for proactive system optimization
relative to the generation of false positives and false negatives, the
internal audit "continuous audit" manager has the authority to change
lter parameterizations as needed. This feedback loop facilitates
continuous process improvement whereby the quality of issued
170
CASE STUDY C: INCREASING AUDIT EFFICIENCY
exceptions improves over time. To ensure system effectiveness, a
comprehensive set of procedures and ratios are relied upon.
In particular, there are 18 procedures or ratios that address detection,
deterrence, nancial losses, and compliance issues.
Figure C-1: Branch Monitoring Process
Some specic procedures examined are check advances, excess in
accounts, overdrafts, cashier out-of-balance situations, federal tax
payment cancelations, and electronic fund transfers.
The success of the SAB branch monitoring initiative demonstrates the
value proposition of continuous auditing. In addition, it offers some
insight relative to the interplay that naturally exists between continuous
monitoring and continuous auditing, and this raises important questions
about CA/CM. For example, CM is historically viewed as a managerial
responsibility whereas continuous auditing is perceived as an auditing
function. However, a single system can be implemented to perform both
CA and CM simultaneously. In this setting, questions might arise relative
to issues such as independence.
POTENTIAL ENHANCEMENTS
SAB’s audit alerts can be integrated into a discriminant function with
weights for the various alerts, thus creating a daily "grade" for each
branch. These grades can then be ranked and used to direct "surprise
audits" for branches with higher perceived risk, and process risks could
171
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
be reected in the weights given to each discriminant variable. The
branches with higher risk weights would likely be audited immediately
while others would be lower in terms of audit priority.
KPIs and/or the variables that comprise them can be changed, and a
communications program can be established to warn and educate
employees about these modications. In addition, evolving risk proles
can be linked with the appropriate sets of audit activities and efforts.
Although this may somewhat reduce the efciency of process
monitoring, it could substantially improve overall process performance
in terms of effectiveness.
CONCLUSIONS
The tradeoffs that created the current audit model have changed with the
advent of business information technology. The costs of information
processing and benets of error detection and value conrmation have
also evolved. The SAB example allows for the verication of values,
value ranges, and ratios for the entire data population. Benets include
much stronger condence relative to the accuracy of values as well as
improvements in overall business operations.
The SAB implementation is an extremely useful CA system that provides
a glimpse of things to come both in corporate business processing
systems as well as the assurance function of the 21st century. Substantial
exibility can be designed into this methodology, and a new set of
systems, analytics, auditor structures, functions, and competencies will
result.
REFERENCES
Aquino, C.E., E. Miyaki, N. Sigolo, and M. A. Vasarhelyi. "A Balancing Act."
Internal Auditor (April 2013): 51–55.
Alles, M., G. Brennan, A. Kogan, and M. A. Vasarhelyi. "Continuous Monitoring
of Business Process Controls: A Pilot Implementation of a Continuous
Auditing System at Siemens." International Journal of Accounting Information
Systems 7(2) (2006): 137–161.
Canadian Institute of Charted Accountants/American Institute of CPAs.
Continuous Auditing. Research Report. Toronto, Canada: The Canadian Institute
of Chartered Accountants, (1999).
Committee of Sponsoring Organizations Internal Control—Integrated Framework.
(2013). Retrieved November, 2014 from www.coso.org/ic.htm.
Elliott, R. K. (1994). "The Future of Audits." Journal of Accountancy, 178(3), 74–82.
——. "Twenty-First Century Assurance." Auditing, 21(1) (2002): 139–146.
172
CASE STUDY C: INCREASING AUDIT EFFICIENCY
Hunton, J. E., and J. M. Rose. "21st Century Auditing: Advancing Decision
Support Systems to Achieve Continuous Auditing." Accounting Horizons 24(2)
(2010): 297–312.
Institute of Internal Auditors. Guide 3: Continuous Auditing: Implications for
Assurance, Monitoring, and Risk Assessment. (2005). Retrieved November 2014
from www.theiia.org/guidance/technology/gtag3/?search=GTAG.
ISACA "The COBIT 5 Framework." (2012). Retrieved November, 2014 from
www.isaca.org/cobit/pages/default.aspx?cid=
1001118&Appeal=SEM&gclid=CKu-1rX- 8MECFUQF7AodSVYA5A.
Jans, M. J., M. Alles, and M. A. Vasarhelyi. "Process Mining of Event Logs in
Auditing: Opportunities and Challenges." International Symposium on
Accounting Information Systems, Orlando. (2010).
Littley, J., D. Minaar, D. Farineau, and R. Soles. "CA/CM: What is Driving
Continuous Auditing & Continuous Monitoring Today?" White paper. KPMG
(2010).
Vasarhelyi, M. A., and F. B. Halper. "The Continuous Audit of Online Systems."
Auditing: A Journal of Practice and Theory 10(1) (1990): 110–125.
Vasarhelyi, M. A., S. Kuenkaikaew, and S. Romero. "Continuous Auditing and
Continuous Control Monitoring: Case studies of technology adoption in
leading internal audit organizations and external audit teams." Working Paper.
Rutgers Accounting Research Center (2009c).
173
CASE STUDY D
Implementing
Continuous
Monitoring at
Vodafone Iceland
Mar
´
ıa Arth
´
ursd
´
ottir
H
¨
orður M
´
ar J
´
onsson
Sindri Sigurj
´
onsson
INTRODUCTION
Vodafone is one of the world’s largest telecommunications companies. It
provides a range of communications services including voice, messaging,
data, and xed communications. With revenue of GBP 43.6 billion at the
end of the 2013–14 nancial year,
1
Vodafone has mobile operations in 26
countries, partners with mobile networks in 53 more, and xed
broadband operations in 17 markets. As of September 30, 2014, Vodafone
has 438 million mobile customers and 11 million xed broadband
customers.
Vodafone Iceland, registered on NASDAQ OMX Nordic stock exchange,
is a quad-play service provider with mobile and xed voice, broadband
Internet access, and IPTV services. Vodafone Iceland was established in
2003 following the merger of three telecom companies. In 2006, Vodafone
Iceland became the rst single brand partner at Vodafone Global, with
full access to the latter’s know-how, ready to market products, marketing
assistance, procurement, and consultancy in networking.
1
http://vodafone.com/content/dam/vodafone/investors/nancial_results_feeds/preliminary_
results_31march2014/p_prelim2014.pdf
175
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
Mar
´
ıa Arth
´
ursd
´
ottir, head of nancial planning and analysis (FP&A) at
Vodafone Iceland, is responsible for the company’s nancial analysis,
budgeting, business intelligence (BI), management reporting, revenue
assurance, billing, and structuring. Arth
´
ursd
´
ottir is also the main driver
in Vodafone Iceland’s project of implementing BI solutions and
continuous monitoring (CM) within the company.
In the often complex telecom business, correct ow of data and data
quality is vital to both employees and customers alike. Employees must
be able to evaluate and make correct decisions on short notice even if
systems and networks don’t work according to plan; it is therefore crucial
that information about Vodafone Iceland’s customers and its services is
correct at all times. Customers in the telecom industry are known to have
limited loyalty to their operators, and because it is relatively easy to
change providers, customer churn is quite high. It is therefore critical for
service providers like Vodafone Iceland to know about errors and/or
discrepancies in the customer relations data processes as soon as they
arise. This enables Vodafone Iceland to resolve these errors quickly and
even proactively suggest new and altered services when appropriate.
Vodafone Iceland’s main focus is to maximize customer satisfaction, as
Arth
´
ursd
´
ottir stresses the following:
We want to keep our customers content and happy. We need to be
sure all customer data records are delivered from the user, through
our network, into our billing gateway and ensure this data ends
up on the customer invoice, correctly and in a timely fashion.
CONTINUOUS MONITORING IN VODAFONE
ICELAND
In 2009, Vodafone Iceland embarked upon a project of designing and
implementing a new business intelligence solution. The company wanted
to improve the efciency of the nancial closing process and at the same
time make nancial information more easily available to the management
team. The company soon discovered that it had too little control over the
quality of the data in the management reports delivered. The reasons for
the low quality of information provided varied between months; in some
instances data got lost on the way, while in others attributes such as new
departments or account numbers were not mapped in a consistent
manner as products and services had been incorrectly set up within
internal systems before they reached the nancial ledger.
Arth
´
ursd
´
ottir observed that many of the internal processes, such as the
preparation of the nancial statements, included a lot of manual work
and re-work (thus increasing the potential for error), resulting in delayed
monthly closing, with work around the clock at the end of every month
176
CASE STUDY D: IMPLEMENTING CONTINUOUS MONITORING AT VODAFONE ICELAND
to discover and repair errors that had occurred during the period.
Arth
´
ursd
´
ottir saw the potential benet of automating the process further,
including continuously detecting and repairing errors as soon as they
occurred, thus shortening the nancial closing cycle, as well as avoiding
peaks of intense work at the end of every month.
Other potential areas of benet using CM were dened by Arth
´
ursd
´
ottir
and her team, such as identifying possible revenue leakage, improving
customer relationship management, streamlining processes such as the
billing process, and monitoring the quality of data owing between
different internal systems and even external third-party systems.
"At Vodafone Iceland we are rating millions of Call Detail Records
(CDRs) per day, for hundreds of thousands of customer services. The
CDRs are being received from dozens of different network elements and
systems. We need to ensure that all these events are handled correctly
and quickly, and validate the integrity of all customer services. We also
have to spot and stop potential fraud in our network," says the
company’s revenue assurance project manager. Consulting with
Vodafone Iceland’s BI service provider, Arth
´
ursd
´
ottir and her colleagues
decided to begin using some new CM software, exMon.
2
The producer of
exMon, Expectus Software, is the only CM technology vendor with a
local presence in Iceland.
Revenue Leakage
In the rst phase of the project, Arth
´
ursd
´
ottir and her team implemented
exMon for revenue assurance. Revenue leakage is a known issue in the
telecom industry. According to TM Forum, "Convergence and lack of
visibility across an ever-expanding value chain are causing growing
revenue losses for Service Providers, as evidenced in a one-of-a-kind
benchmark study conducted by The TM Forum. The most surprising
breakthrough was the tangible proof that Service Providers incur an
average of one-percent revenue leakage with a maximum recovery of 50
percent."
3
The study’s authors also note, however, that collected data
across ve continents indicates that prevention really works and that
service providers that validate a large percentage of their data see
signicantly lower revenue leakage.
This was also the case with Vodafone Iceland. The company tries to
minimize revenue leakage where possible, but because it is not a large
organization, operating a dedicated revenue assurance department is not
nancially feasible. The company evaluated the risk involved and used
automated CM for situations where the stakes were assessed as high,
gradually working their way down the list to less important items.
2
http://exmon.com/cm/#home
3
www.tmforum.org/TMForumPressReleases/RevenueAssuranceStudy/36002/article.html
177
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
One example of revenue leakage identied involved blocked accounts
due to debt. When a customer is terminated or blocked (that is, his or her
account sent to a debt collection agency), it is important that he or she is
no longer able to use the service, and blocking the account requires
actions in many systems. Blocked accounts are now under continuous
monitoring and if something in the closing process has failed, the system
will ag usage on the terminated telephone number.
Another example showed certain Internet connections or xed lines not
being charged to the retail customer, yet being charged to Vodafone
Iceland by their backbone supplier. By analyzing the exceptions, the
origin of the problem in the workow was identied—in this case a
software bug—and that information was delivered to software engineers
who xed the problem. The process owner continues to monitor this via
CM, should a similar exception arise again.
If not managed correctly, the complexity of discount business rules and
price changes can also result in revenue leakage. Vodafone Iceland
therefore also uses CM to detect failing discount rules in the systems that
result in missing discounts and/or illegal discounts being issued by
employees. The execution of price changes is also being monitored
closely, ensuring that tariff price changes are always correct. The system
regularly compares (daily or monthly) all price changes against the
"golden copy" owned by the company’s marketing department to
monitor and verify all price changes implemented on the billing system.
Process of Monthly Financial Closing
As previously mentioned, the initial goal Arth
´
ursd
´
ottir and her team
decided to address early on in the project was to decrease the time
required to process the nancial closing each month. This challenge was
resolved through two key initiatives—improved BI capabilities using
Microsoft Business Intelligence solutions and implementing CM checks
with exMon at various points in the closing cycle. Using these two
methods, both management and analysts were able to analyze their
nancial information in a much more comprehensive manner, as
automated checks were performing their detective work behind the
scenes at all times, ensuring accuracy throughout the process.
After several months of trial and error and ne-tuning of monitoring
check points, the Vodafone Iceland team was experiencing exceptions
being detected and xed on a daily basis. This dual approach of
addressing management reporting requirements through state-of-the-art
BI solutions and ensuring underlying data quality through means of CM
resulted in the nancial closing process now being nalized within hours
instead of days.
178
CASE STUDY D: IMPLEMENTING CONTINUOUS MONITORING AT VODAFONE ICELAND
The Billing Process
An example of the data ow monitored by exMon is that of customer
data, from rst entry through networks and IT systems, into correct
invoices being sent out from the billing system (gure 1). To facilitate
this, a mobile user is registered in the customer relationship management
(CRM) system along with the chosen product and tariff plan. This
registration needs to be delivered to the home location register (HLR) in
the mobile network, giving the customer access to the services he or she
should be able to use (make and receive calls, send SMSs, use data via
Internet, and use data roaming abroad). The customer also needs to be
registered correctly in the billing system, with the same tariff plan and
potential value-added services and discounts.
Figure D-1. Example of How Customer Data Flow Is Checked Within
and Between Many Systems to Ensure Correct Billing
Customer charging records can fail, rendering the billing system unable
to process the records correctly for them to be billed to the customer. Such
errors may include a missing link between the customer’s IP address and
the customer usage data, or trafc on a cell phone number not recognized
by the billing system. By monitoring these and other similar issues in an
organized manner, receiving lists every week with errors, spending time
analyzing and trying to nd out the root cause instead of focusing on the
symptoms, the billing department at Vodafone Iceland observed a 74
179
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
percent drop of billing data processing errors within a period of 12
months (gure 2).
Figure D-2. Monitoring Exceptions in the Billing Process Resulted in a
74 percent Drop in Billing Data Processing Errors Within 12 Months
Other CM checks implemented to ensure end-to-end reconciliation of
CDRs from network elements to billing include
r
ensuring that all CDR les have been delivered from network
elements to the billing system;
r
reconciling that the all CDRs within each le have been rated in to
the billing system;
r
ensuring that all CDR les are being delivered in a timely fashion;
r
reporting on potential mediation or rating errors in CDR-based
erred events;
r
reporting on missing subscribers or subscriber services based on
erred events; and
r
reconciling all rated records against the customer bill.
All exceptions identied the exMon CM system enables easy follow-up
and handling through a web user interface. Within this portal, cases are
assigned to the responsible parties, who can actually x the root cause of
the problem as soon as it is identied and prevent exceptions from
reoccurring. The follow-up function immediately gives the relevant
teams and employees a clear overview, enabling them to x the problem
without having to look into other systems or databases. The system logs
all actions and escalations, and the responsible person can assess the age
of exception cases, status, and level of severity at a glance.
180
CASE STUDY D: IMPLEMENTING CONTINUOUS MONITORING AT VODAFONE ICELAND
Fraud Monitoring
Fraud detection was a high priority in the process of implementing CM.
Fraud cases can be of different origin and can cost both the customer and
the service provider a lot of money. Fraud can arise within companies
through various means, such as by employee abuse of access to systems
or nancial resources. Fraud can also originate from outside the
company. A common example in the telecom sector is abuse of SIM cards
from stolen cell phones, where they are used to generate usage to
premium numbers and produce revenues to third parties. Another
example of external fraud is a break-in into a customer’s IP network to
generate high trafc to servers in some foreign countries. It is vital to be
able to detect and stop fraud being committed in the company’s systems
as quickly as possible. Investment in an expensive specic fraud
management system has not been an option for small- to medium-sized
enterprises (SME) like Vodafone Iceland, so they use exMon to monitor
certain patterns of behavior for potential fraud. Examples of potential
fraud monitored continuously include break-ins into business telephone
systems, roaming fraud, SMS spam, fraudulent use of all inclusive
packages, and credit card fraud.
Customer Relationship Management
After the initial phases in the project of focusing on the revenue leakage,
fraud, and billing process errors, Arth
´
ursd
´
ottir and her colleagues turned
their attention toward how they could use the CM process to enhance the
quality of their CRM. According to the head of customer care at Vodafone
Iceland, CM of customer relations has enabled her team not only to
reactively repair things that go wrong, but also to proactively contact
their customers with specic advice on how to get more value out of their
service plans. This use of CM checks to detect cross-selling opportunities
and areas where the company can add real value to its customers’ usage
of communication services is extremely innovative.
First, some examples of reactive repair results of monitoring the customer
relationship and use of their services:
r
If the mobile subscriber has a family subscription, then every
family member needs to be linked to and registered to the same
account so that each will receive the right benets and discounts
they have been promised. This process is now being monitored
and a discount check made on the billing data.
r
If a mobile subscription is paid by a subscriber’s employee, then it
is vital that the subscriber is set up in the correct customer user
group, because usually subscribers within a company are allowed
181
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
to make free calls within a dened group of users. Today, the
correctness of this process is secured by an automated check.
r
Vodafone Iceland used to have a problem with sometimes
overcharging customers who were switching from ber optic cable
Internet price plans to "Fiber to the Home" (FTTH) price plans. The
company had to issue credit notes every month because customers
were being double billed, sometimes even for months. Vodafone
Iceland therefore introduced CM checks. These identied the
problem as system failure within the termination process. The
problem was handed over to the software engineers, and the
system was xed. The company thus saved signicant time on
reactive corrections and reduced customer calls and, most
importantly, saved the customer from irritation and inconvenience.
Vodafone Iceland’s customer care department is now able to take
proactive measures, including that of informing the customer in almost
real time when he or she is getting close to his or her maximum amount
of data, SMSs, and voice minutes included in his or her price plan. The
message directed at each customer then includes information about
either how to block further usage, like when a parent wants to prevent a
child from further downloading from the Internet, or how to
economically buy additional download or minutes to keep using the
service. Customers’ usage of mobile voice and Internet services abroad is
also being monitored continuously to prevent "bill shock" when they
roam into expensive international rating zones.
These customer-related measures in CM are very important to Vodafone
Iceland, for which the main focus area is customer satisfaction. Thanks to
closer monitoring of customer-focused processes, the company can now
report tangible results in that area.
Culture Change and Enhanced Quality
of Work Flow
The project of implementing CM within different departments and units
has resulted in signicant positive changes throughout Vodafone Iceland.
There is a growing culture of proactively implementing checks in
different areas when developing and deploying new processes and
services. People are more amenable to and proactive about checking their
own work, which results in a culture that is more proactive and
preventive. The process has also resulted in an enhanced visibility of
internal processes. More people think about the entire process, instead of
just their part of it, and are now used to drawing up process maps,
discussing them and internalizing them. Responsibilities in every step of
these processes are more visible than before and the process of nding
the root cause(s) behind each failure is now much shorter.
182
CASE STUDY D: IMPLEMENTING CONTINUOUS MONITORING AT VODAFONE ICELAND
This enhanced sense of initiative has spread to customer relationship
management and the culture of proactive customer care is now quite
visible. Customer care agents are more conscious about preemptively
detecting errors before they reach the customer and then proactively
giving advice to their customers about better and more economical ways
to use the service they are paying for.
Today, Vodafone Iceland employees really see and appreciate the value of
CM as it has enabled them to prioritize in a more correct manner and
focus their energy, time, and skills on the right issues by reducing or
eliminating repetitive manual work on problems that used to repeat
themselves daily.
CHALLENGES AND LEARNING
There were some challenges associated with implementing CM in
Vodafone Iceland. One of these was unclear ownership of processes in
the early phases of CM implementation. Implementing CM has helped
Vodafone to map and assign ownership to various processes and work
ows. Another challenge was to correctly assess the value of each check
created. The subsequent alarm and analysis cycle can easily become an
unnecessary distraction instead of a benet if the initial assessment of the
check is not thoroughly completed. It has also been important to align the
frequency of alarms and exception lists with the human resources
available and the time required to analyze and x the problems. Working
with the system for several months has also demonstrated that it is
important to remove checks that are no longer relevant and keep the
overall set up of checks up to date.
THE FUTURE
The goal of Vodafone Iceland is to expand the use of CM across the entire
organization. There are still departments and areas of operation that have
not been introduced to the system. Arth
´
ursd
´
ottir sees opportunities in
enhancing and optimizing the use of the system, and in extracting and
analyzing statistics from the use of CM, discovering patterns and new
dimensions of potential value. PM is taking over an increasingly greater
portion of external audit of the company. Several manual checks
completed in previous years through computer security audit, have now
been canceled and are instead performed automatically on an on-going
basis. A process of presenting an overview of checks to external auditors
has been in operation for two years. The external auditors then audited
selected checks in the CM process. This development will continue and is
183
AUDIT ANALYTICS AND CONTINUOUS AUDIT:LOOKING TOWARD THE FUTURE
expected to increase trust in the quality and ow of data within the
company.
CONCLUSION
Overall, Vodafone Iceland’s experience with CM has been a good one.
The company operates within a complex set-up of networks, systems and
services, as is usual with quad-play communication service providers.
Their conclusion is that CM is of great benet to complex operational
environments like those in the telecom sector, and that the journey of
using the system is just beginning. Arth
´
ursd
´
ottir states:
We have high expectations about expanding the use of the CM
system. Today, the main emphasis of the company is to build a
trustworthy long-time relationship with our customers.
Companies are increasingly realizing that investment in customer
care pays off, whereas ever increasing acquisition cost per
customer doesn’t. We see opportunities in using CM in different
ways within analyzing customer behavior and customer account
data. We still have a long way to go to map up possible ways to
use exMon to increase the mutual value of the relationship with
the customer, and we expect to discover new areas of usability in
the future. Our goal is to steadily enhance the quality of our
business processes and establish a positive cycle of renewing the
set of checks.
184
AUDIT
ANALYTICS
aicpa.org | cpa.com
AICPA Assurance Services Executive Committee
The mission of the AICPA Assurance Services Executive Committee (ASEC)
is to assure the quality, relevance, and usefulness of information or its
context for decision-makers and other users by (1) identifying and prioritizing
emerging trends and market needs for assurance, and (2) developing related
assurance methodology guidance and tools as needed. ASEC achieves its
mission by:
providing guidance and leadership in identifying and prioritizing
signicant emerging assurance trends and market needs while engaging
users, preparers, and inuencers toward action;
developing assurance guidance by creating suitable criteria when
necessary, and/or performance guidance, as appropriate;
communicating new assurance methodologies, guidance, and
opportunities to our members and the profession on a global basis; and
creating alliances with industry, government, or other specialized groups
to improve CPA access to new assurance opportunities.
For additional information on the AICPA’s Assurance Services Executive
Committee please visit aicpa.org/ASEC.
AICPA Business Reporting, Assurance and Advisory Services Team
The overarching role of the AICPA’s Business Reporting and Assurance &
Advisory Services Team is to provide leadership oversight, direction and
visioning for emerging business reporting and assurance issues and initiatives
that are identied and addressed through input from AICPA members,
committees and staff.
For more information on the Business Reporting, Assurance and Advisory
Services Team initiatives, please visit aicpa.org/AAServices.
CONTINUOUS
AUDIT
AUDIT
ANALYTICS
AUDIT ANALYTICS and CONTINUOUS AUDIT
and
AUDIT
ANALYTICS
AUDIT
CONTIN UOUS
Looking Toward
the Future
17970-344_Audit Analytics_final.indd All Pages 7/9/15 10:14 AM
