Leveraging surveys in internal audit.

Surveys are effective for reviewing intangible topics such as the ethical environment, the corporate culture, and entity-level controls, which require an examination of soft controls like integrity, values, accountability, leadership, openness, empowerment, and competence. These soft controls and other control environment elements are complex and arguably some of the most difficult to assess.  

COSO’s Internal Control - Integrated Framework (IC-IF) draws attention to the vital role that the control environment and entity level controls play in establishing and preserving a strong internal control environment. Internal auditors are expected to evaluate the organization’s tone at the top, management’s operating style, and risk management philosophy. While reviewing these subjects often pose unique challenges, with data internal auditors can make appropriate recommendations to improve the organization’s governance and promote appropriate ethics and values within the organization. 

Surveys should be conducted like projects so good planning and preparation reduce the likelihood that key steps will be missed and avoid wasting time and money. To conduct surveys effectively, the following stages are critical:

  1. Set Objectives: A successful survey begins with the establishment of clear objectives that define what will be accomplished. This will guide information collection and establish the baseline for everything from the resources needed to the amount of time the survey will last until completion. Conversely, when objectives are not clear, surveys lack focus, ask confusing, vague or unrelated questions and in the worst-case scenario become a costly waste of time and money. Always ask, what exactly do we want to find out?

  2. Design the survey: This is a good time to apply the maxim “An ounce of prevention is worth a pound of cure”.  The following questions are essential to design a survey effectively:

    • How many people should be polled to feel comfortable acting on their responses?

    • Who will be included and who will be excluded from the survey?

    • How is the data going to be collected and tabulated?

    • What will be done with the data after getting it?

    • What resources (e.g., money, time, skills, tools) are available to pursue the objectives?

    • Do we have the support from senior management to do this? 

    • What can go wrong and what needs to go right?

  3. Prepare the questionnaire: Make sure to use clear and simple language. Match what information you are seeking against the amount of time available to find out. You should not conduct a larger-than-necessary survey asking myriad questions because the response rate and the quality of data will decrease with longer questionnaires. Close-ended questions are easier to tabulate and analyze, but open-ended questions should be considered too because they provide insightful feedback from survey participants.

  4. Administer the survey: While running the survey, watch for unusual and unexpected situations developing and be receptive to informal feedback from participants. Treat the survey with a sense of urgency, giving reasonable but short deadlines for completion. Have a high-ranking officer in the organization write the cover letter or announcement, and have a process in place to send reminder notices. Use blanket reminders if anonymity is more important than response rate; otherwise, you can target your reminders.

  5. Manage and analyze survey data. This involves collecting, tabulating and reviewing the data. When conducting a survey, you are likely to notice that data starts coming in shortly after the questionnaire is distributed, and the number of responses decreases over time. This is another reason not to give participants too much time. If they don’t complete it immediately, many of them will not do it later. As the completed questionnaires start coming in you can start to enter the information into your spreadsheet or database. It will help you identify issues with the data if there are any and decide how they should be addressed. Don’t be intimidated by statistics because you are likely to find out what you need by using simple calculations. Just remember to ask these two key questions during the analysis. 1. What is the overall score for each question and 2. What is the score among {subgroup}? The results can be plotted on graphs and charts.

  6. Report the results. The last step is reporting the results, which also includes making presentations of the methodology, results, analysis, and conclusions. Reports often include charts, tables, and graphs and most use colors liberally. The objective is to communicate the results clearly and concisely.

Surveys are effective for reviewing intangible topics such as corporate culture, and entity-level controls, which require an examination of soft controls like leadership, openness, empowerment, and competence. #audit Click to Tweet

Tips for Designing Effective Surveys

  1. Avoid lengthy questionnaires, keep the questions simple, and the response options logical.

  2. Avoid jargon, abbreviations, very technical, colloquial or slang terms. 

  3. Plan and design appropriately so the survey can be repeated in the future to compare results and perform trend analysis.

  4. Avoid negative questions such as “Did your manager fail to give you a performance evaluation last year?”

  5. Avoid confusion with Agree/Disagree items.  For example: “Employees should not be required to work excessive overtime or travel on weekends.” 

  6. Decide carefully when to use closed or open-ended questions. 

  7. Phrase your question clearly when asking open-ended questions.

  8. Learn the characteristics of the survey’s targeted respondents, so the response categories make sense to them.

  9. Questions that rely on time periods should be specific and defined clearly, ideally instructing respondents what units of time to use.

  10. Run the questionnaire with the endorsement of senior management

  11. Analyze the data without overcomplicating the process with excessive statistics.

  12. Whenever possible, share the information with employees, so they know that you valued their input and that something was done with the results.  They will be far more willing to complete the next survey you send to them.

Surveys make it possible for internal auditors to collect large amounts of data. When used as a complement to other sources of evidence, the result is better and more authoritative evidence to evaluate objective and subjective topics.

Interested in learning more about this and other tools and techniques? Join Dr. Murdock when he teaches Lean Six Sigma Skills for Auditors, Internal Audit School, and High-Impact Skills for Developing and Leading Your Audit Team.

 

Photo by Samuel Zeller on Unsplash